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EXECUTIVE SUMMARY 



In 2004, the Lumina Foundation for Education approved a generous grant to support 
validation research to explore and document the validity of the Community College Student 
Report (CCSR), add to the higher education field’s understanding of student engagement, and 
help to identify research or institutional practices that require further attention. The study was 
conducted in three strands that linked Community College Survey of Student Engagement 
( CCSSE ) respondents with external data sources: (1) data from the Florida Department of 
Education; (2) data from the Achieving the Dream project; and (3) student record databases 
maintained at community colleges that have participated in the CCSSE survey and are either 
Hispanic-Serving Institutions or members of the Hispanic Association of Colleges and Universities 
(HACU). All participating students had participated in the 2002, 2003, or 2004 administrations of 
the Community College Student Report, CCSSE’s survey instrument. 

The Florida data set contained complete records of students’ demographics, placement 
tests, course taking, and completion points. This data source was analyzed by a team at the 
National Center for Higher Education Management Systems (NCHEMS), directed by Peter Ewell. 
The Achieving the Dream data source consisted of extensive demographic data and term-level 
records from colleges participating in the national Achieving the Dream initiative. This data 
source was analyzed by Derek Price of Praxis Associates. The Hispanic Student Success data 
source was compiled by obtaining transcript data from participants in a CCSSE HSS consortium 
and other HSI and HACU colleges. This data source was analyzed by Greg Smith, an 
independent consultant. 

Florida Study Results 

The pattern of results obtained from the Florida study broadly confirms positive 
relationships between the construct of student engagement as measured by CCSSE and 
community college outcomes. CCSSE benchmarks and item clusters show a consistent pattern 
of significant association with academic outcomes like GPA, degree completion, and attainment 
of important academic milestones, after controlling for student characteristics and entering ability. 
The strongest of these net effects emerged where they should most be expected — for “academic” 
areas of engagement such as Academic Challenge, Active and Collaborative Learning, Student- 



5 




Faculty Interaction, and Mental Activities. Self-reported academic Gains on CCSSE also are 
significantly related to actual academic achievement measures, both directly (confirmed through 
bivariate correlation analysis) and after controlling for student ability and background. This 
finding helps validate CCSSE s use as a “proxy” measure for student academic achievement. 
While pervasive significant net effects are less typical of behavioral measures of student success, 
such as persistence to a second term or persistence to second year, they do occur repeatedly 
across both longitudinal cohort datasets. Moreover, the CCSSE benchmarks and item clusters 
that emerge as significant in these cases are those that the retention literature says should do so: 
Support for Learners, Student Services, and occasionally, Collaborative Learning. 

Achieving the Dream Study Results 

This Achieving the Dream study yielded mixed results. The most promising results were 
for academic achievement (cumulative GPA) and persistence (credit completion ratios and fall-to- 
fall retention). Less promising were the results when predicting course completions across 
developmental math, writing and reading, as well as college-level algebra and English. The 
Achieving the Dream study also examined engagement levels for low-income students, minority 
students, and students exhibiting known risk factors, and found that in each case these students 
were more engaged than a comparison group. Overall, Active and Collaborative Learning is the 
most powerful and versatile of the five CCSSE benchmarks when predicting student success for 
Achieving the Dream colleges using several different outcome measures. 

Results of the HSS Study 

In the HSS study, the student engagement scales were predictors of both CCSSE self- 
reported outcomes and transcript-derived student outcomes. Overall, two student engagement 
scales — Academic Challenge and Support for Learners — were the most consistent predictors of 
student outcomes. After considering the effects of student engagement, when self-reported 
academic Gains and satisfaction were added as either independent variables or moderator 
variables, self-reported Gains tended to add little to our ability to predict outcomes, whereas 
satisfaction makes an independent contribution. Immigrant status should definitely be accounted 
for in any future CCSSE research. Immigrant students reported much higher levels of Student 
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Effort, Academic Challenge, Support for Learners, and Academic, Personal Development, and 
Vocational Goals Gains than did non-immigrants. 

Overall, results clearly demonstrate that in assessing the validity of the CCSSE, the 
choice of student outcomes variables is very important. The analyses accounted for larger 
proportions of variance in cumulative GPA, total credit hours completed, and average credit hours 
than in first to second term persistence, first to third term persistence, and number of terms 
enrolled. Further, depending on the student outcome of interest, some CCSSE self-reported 
outcomes seemed to be good proxies for transcript-derived outcomes, specifically cumulative 
GPA and total credit hours earned. Overall, many of the CCSSE variables, as well as 
corresponding derived scales and factors, demonstrated solid relationships with both self- 
reported and transcript-derived student outcomes. 

Overall Results 

The results of these studies point to the following overall conclusions: 

• There is strong support for the validity of the use of the CCSR as a measure of 
institutional processes and student behaviors that impact student outcomes. The 
strength of the results is derived from strong consistency across three studies using 
virtually independent samples and analyzed by three different analysts. 

• The studies confirm a long tradition of research findings linking engagement to positive 
academic outcomes. The significance of this research is that it was conducted on 
community college students who have been markedly understudied relative to students in 
baccalaureate-granting institutions. 

• There is strong consistency in the relationship between engagement factors and outcome 
measures across the three studies; however, some outcomes have stronger relationships 
to engagement than others. 

• The Support for Learners benchmark was consistently correlated with measures of 
persistence. While the majority of the CCSSE items were acquired from the National 
Survey of Student Engagement, several items in the Support for Learners benchmarks 
are unique to the CCSR and were intended to assess issues related to persistence. 
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INTRODUCTION 



The central purpose of this research was to explore and document the validity of the 
Community College Student Report (CCSR), which is the instrument used by the Community 
College Survey of Student Engagement (CCSSE). In addition to providing important validation of 
the CCSR and its use as a measure of institutional effectiveness, the studies make a significant 
contribution to the literature on student engagement. Despite the voluminous empirical literature 
on the positive impact of quality and effort of work on academic success (Pascarella & Terenzini, 
2005), there has been minimal investigation of the impact of student engagement in samples of 
community college students. Attempts to quantify the proportion of higher education literature that 
utilize community college samples consistently estimate the proportion of literature on community 
college samples at 10% or less. Pascarella (1997) acknowledges that at most 5% of 
approximately 2600 studies reviewed in the seminal text that he co-authored with Terenzini ( How 
College Affects Students, 1 991 ) focused on community college students. Cofers and Somers 
(2000) report that in their search of the Education Resources Information Center (ERIC) 
database, 10% of the nearly 2000 publications on college persistence included two-year students. 
A systematic examination of five major higher education journals found that only 8% of articles 
mentioned community colleges (Townsend, Donaldson, & Wilson, 2004). A meta-analysis 
examining support for Tinto’s (1 993) theory of retention, using only studies conducted with 
community college students, found only six studies that qualified for inclusion in the analysis after 
a literature search of three major databases (Wortman & Napoli, 1996). These findings strongly 
indicate that student engagement is one of the more poorly studied areas within the community 
college literature. Thus, the empirical higher education literature, particularly the literature on 
student engagement, have overwhelming focused on students at baccalaureate-granting 
institutions, leaving a gap in the literature on community college students. 

This paucity of empirical literature has resulted in some recent high-profile statements 
noting the lack of empirical evidence for student integration or engagement models in research 
utilizing community college students. A review of community college research (Bailey & Alfonso, 
2005) found that the quantity and quality of research on community college institutional practice 
inadequate. A review of the empirical evidence for Tinto’s (1993) theory of student departure 
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found that there are notable differences in the theory’s support between the two- and four-year 
sectors (Braxton, Hirschy, & McClendon, 2004). These statements reflect the lack of empirical 
work done using community college samples, not empirical work demonstrating a lack of 
applicability of student integration or engagement models. Thus, the research conducted in 
support of this grant directly fills a gap in higher education literature. 

The purpose of the inquiry was to establish links between results obtained on the 
Community College Student Report (CCSR) and a variety of education outcomes. The CCSR is 
fundamentally designed to measure the processes — institutional practices and student 
behaviors — that lead to higher levels of learning and educational attainment. As such, there is an 
implicit assumption that engagement in effective educational practices has a positive impact on 
outcomes. To establish that the CCSR measures processes that matter, survey responses were 
linked to a variety of short- and long-term outcomes. The strategy of conducting three parallel 
studies enhances the power of this research by allowing us to examine results across studies and 
identify areas of convergence and divergence. Furthermore, because three different consultants 
conducted three separate strands of the research, the variety of analytic approaches used by the 
consultants provides multiple perspectives for examining and understanding the data. 

Each of the analysts was supplied with derived constructs for data analysis. These 
variables are described in detail in Appendix D. The constructs consisted of CCSSE 
benchmarks, engagement item clusters, and gain item clusters. The development of the 
benchmarks and the engagement item clusters are described in detail elsewhere (Marti, in press). 
It should be noted that the benchmarks and engagement item clusters are non-orthogonal; 
engagement item clusters use largely the same items that comprise the benchmarks but contain 
a larger number of item clusters, or factors, that represent a finer grained examination of 
engagement items. The gain item clusters represent three groups of self-perceived gain items in 
academics, personal development, and vocational goals. 
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VALIDATION STUDIES 



Study 1 : Florida Community College System Validation Study 

Study Description 

Sample Overview 

Students enrolled in the Florida Community College System (FCCS) institutions who took 
the CCSSE in 2002, 2003, and 2004 were matched with all term enrollment records provided by 
FCCS for the period fall 1996 through summer 2005. There were a total of 4,823 students who 
completed the CCSR in a primary CCSSE sample and provided an ID that could be matched to a 
record in the Florida Department of Education’s database. Students taking the CCSSE in 2002- 
2004 were more likely to have entered a Florida community college for the first time in recent 
years. More than half of those students (58.8%) included in Long cohort files, for example, began 
their study at FCCS in 2001 or 2002, with only 1 3.3% beginning in fall 1 998 or earlier. This 
means that most of the students in these cohorts have not experienced more than ten to fifteen 
terms of potential enrollment. Students completing the CCSSE — and thus eligible for inclusion in 
the study — also tend to be fairly traditional when compared to others enrolled in FCCS colleges. 
To assess how representative this study sample was, comparative statistics on all entering 
freshman were obtained from the FCCS. Comparisons are presented in Table 1 . 

These differences reflect the kinds of response biases typical of student surveys and, 
more particularly, experienced by most colleges when they administer CCSSE. No attempt was 
made to correct for them in any of the analyses undertaken, and because the most important 
analyses were multivariate, the primary point of interest was the relationships among variables in 
any case. But it is important to point out that the universe of students within which validation was 
attempted differs in a few notable ways from the parent student population. 

Data Construction 

FCCS maintains comprehensive records for all students enrolled for credit in the 28 
community colleges in the state. These records include descriptive data on student 
characteristics, data on basic skills and placement levels, and transcript-level detail on every 
class taken by every student; furthermore, they have been collected under common definitions for 
a very long period of time. 
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Table 1 



Comparison of CCSR Analysis Cohorts with FCCS Population 



Variable 


Actual 

FCCS 

Entering 


Merged 

Cohorts 


Short 

Cohorts 


2003- 
2004 Yr. 


Gender (%) 


Female 


61.6 


60.9 


59.6 


63.2 


Male 


38.4 


39.1 


40.2 


36.8 


Ethnicity (%) 


Asian 


3.0 


2.5 


2.4 


2.5 


Black 


17.8 


13.3 


12.6 


13.7 


Hispanic 


20.0 


13.6 


12.8 


10.7 


Indian 


0.4 


0.4 


0.5 


0.5 


White 


56.7 


68.5 


69.9 


65.9 


Not Reported 


2.1 


1.6 


1.8 


2.0 


Age (%) 


17 or less 


6.4 


25.0 


21.3 


17.9 


18 to 21 


39.7 


53.3 


55.6 


43.0 


22 to 25 


17.6 


6.5 


7.3 


9.2 


26 to 35 


19.4 


6.7 


8.7 


10.9 


36 to 45 


10.7 


5.2 


4.8 


6.0 


46 to 55 


4.8 


1.4 


2.0 


2.4 


Over 55 


1.3 


0.1 


0.1 


0.3 


College Status (%) 


First Time 


76.0 


83.4 


81.0 


78.3 


Transfer 


24.0 


14.1 


16.9 


18.4 


Enrollment Status (%) 


Full-time 


32.6 


46.5 


50.8 


54.4 


Part-time 


67.4 


53.5 


49.2 


46.6 


Goal for Attendance (%) 


AA 


42.6 


58.9 


59.3 


54.6 


AS/AAS 


18.2 


17.9 


17.9 


19.2 


Certificate 


2.2 


1.8 


1.2 


2.3 


Other 


37.0 


21.4 


21.6 


24,2 



In addition to data availability and high quality data, the Florida Community College 
System has other advantages for a study that systematically examines patterns of student 
success. It has a common course numbering system that helps to ensure that basic skills and 
“gatekeeper” courses are of equivalent content across campuses. Perhaps most important, 
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common placement standards and a common placement test (the Florida CPT) provide standard 
measures of entering student ability that can be used as a control variable for studies of net 
effects. This is an unusual and valuable property in a community college dataset. 

Unit record data drawn from the records system of the FCCS were supplied to NCHEMS 
by CCSSE in the form of individual SAS files containing discrete bodies of related variables. 
These records contain individual entries for each student for each term of attendance at a Florida 
Community College for all students enrolled in the period fall 1996 through fall 2005 who had also 
completed the CCSSE instrument in 2002, 2003, or 2004. CCSSE data were supplied directly in 
the form of a single SAS file. All records were individually identifiable through a student 
identification number that was used to construct analytical files. 

NCFIEMS staff converted discrete data files obtained from the Florida Community 
College System to SPSS files and ran basic statistics to verify their contents, ranges, coding 
structures, and similar properties to help determine which data elements would be used. Many 
data elements were eliminated from consideration because they contained only fragmentary data 
or were irrelevant to the validation analysis. Usable and relevant data elements were then used 
to construct a set of analytical files, using the student identification number as the key link. 

Several analytical files were created to support the analysis. 

Long cohort files. Long cohort files were constructed for each fall and spring term 
beginning with fall 1996 through fall 2002. Student’s first term of academic history is first 
determined in these files, and the students are tracked from that start point through the summer 
of 2005. The purpose of these files is to support analyses of long-term patterns of student 
success including remediation success, persistence, and program completion. These files 
contain a “fixed" body of data on each student, including demographic and educational 
background data elements, together with multiple term records containing information about the 
details of enrollment and academic performance. CCSSE benchmarks and scales were included 
in each file. Initial exploratory analyses indicated that there were few differences in student 
behavior across cohorts over time, so all cohorts were merged to maximize the number of cases 
available for analysis. This yielded a total of 1958 usable cases for analysis. 
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Short cohort files. Short cohort files were constructed for each fall and spring term 
beginning with fall 1996 through fall 2004. These files were constructed in the same manner as 
Long cohort files but containing only three terms of academic history. These files were created 
because many students for whom records were available could not be included in Long cohorts 
because they began their studies more recently than the fall of 2002. 1 These cohorts were used 
to examine more immediate student outcomes such as second term persistence, first-year GPA 
and course completion, and success in remedial and gatekeeper courses. Short cohort files 
contained a total of 2658 usable cases. 

Cross-sectional performance file. This file contains all students, regardless of level, 
enrolled in the period fall 2003 through summer 2004 (Academic Year 2003-2004). This file was 
created to correspond to a substantial administration of CCSSE in the spring of 2004 and 
represents the largest pool of students available for these validation analyses (N = 5468). For 
most questions on the CCSR, students are specifically asked to report their perceptions and 
experiences during the “current year,” and this period corresponds to that year. Because 
students contained in this file are at different stages in their academic careers, this file cannot be 
used to examine outcomes like persistence or program completion. But it is the largest and 
probably most appropriate universe within which to examine the link between CCSSE self-reports 
and immediate academic outcomes such as GPA and course completion. 

Course-taking files. These files contained all courses taken by students and were 
aggregated into a longitudinal record to examine student success in “gatekeeper” courses, basic 
skills courses, fulfillment of general education requirements, and so on. Because of their size, 
these files were maintained separately and were merged into cohort files as needed for particular 
analyses. 

CCSR files. Complete CCSR responses were maintained as a separate file containing 
data from both the 2002 and 2004 administrations. These were merged with the analytical files 
as needed to examine particular items and to provide additional control variables for student 
characteristics not included in the Florida Community College System records. 



1 The largest of the three administrations of CCSSE in Florida occurred in the Spring of 2004. 
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Following standard NCHEMS procedures for conducting longitudinal student flow 
analyses, Cohort Files consist of a single block of “fixed” data elements containing information on 
student demographics, educational background, and initial enrollment status, followed by multiple 
term records containing information on the specifics of enrollment for each student for each 
subsequent term. Figure 1 shows the basic structure of all cohort files created. 

Study Variables 

A list of all data elements in the cohort files is provided in Appendix E. These data 
elements were identical for Long and Short cohort files; the only difference between the two files 
was the number of terms for which data were provided. Most data elements in the cohort files 
were taken directly from student records, but some (e.g. age) were derived from existing data 
elements. Some additional control and student selection variables were obtained from CCSSE 
responses. Derived data elements are flagged with an asterisk in the list. 

Dependent variables for the validation study consisted of a range of performance 
measures defined longitudinally by relating two or more “milestone events” in a given student’s 
enrollment history within a given period of time. For example, the Three-year Degree Completion 
Rate relates a given student’s achievement of an associate degree at a Florida community 
college with his or her first credit enrollment in a Florida community college within a three-year 
time period. As another example, the “transfer-ready” rate for skills-deficient students relates the 
point at which a given student is placed below college level in one or more basic skills with his or 
her achievement of “transfer-ready” status, regardless of whether or not he or she has earned a 
credential. An illustrative chart of “milestone events” of this kind is presented in Figure 2. 
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Figure 1 

Cohort File Structure 
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Figure 2 

Milestone Events in a Student Enrollment Pathway 
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These performance measures recognize the fact that that such “milestone events” may 
occur in different orders for different students. For example, students may enroll for their first 
college-level credit at a point either before or after their enrollment in a developmental course. 
Similarly, students may transfer before or after they have earned a credential or achieved 
“transfer-ready” status. Each performance measure is calculated independently in this manner 
within a given analysis. The basic performance measures prepared for the validation study are 
as follows. 

Completion rate. Students who earned an associate degree, tracked from the point at 
which they enroll for the first time for credit leading to a degree. Students placed in 
developmental work are considered to have reached this start point if they are enrolled in the 
appropriate course of study. 

Second term persistence rate. Students in an entering cohort that remained enrolled in a 
program leading to a credential or a degree at any Florida Community College the following term, 
tracked from the point at which they enroll for the first time in instruction that leads to a credential. 

Second year persistence rate. Students in an entering cohort that remained enrolled in a 
program leading to a credential or a degree at any Florida Community College the following year 
(fall for fall-term starters, spring for spring-term starters), tracked from the point at which they 
enroll for the first time in instruction that leads to a credential. 

College pathway status. College pathway status is achieved when the student has 
completed 1 2 semester hours (or equivalent) of college credit, and can therefore be considered to 
be seriously on the path toward achieving a college credential. 

Transfer-ready status. Transfer-ready status is achieved when the student has (a) 
completed 30 SCFI of college credit; (b) has passed or placed out of all developmental work and; 
(c) has completed English Composition, a college-level math course, and one college-level 
course in each basic discipline cluster (science, social science, and humanities). 

Cumulative grade point average. Cumulative grade-point average that was computed as 
earned in all completed courses. 

Cumulative credit completion ratio. The total number of credit hours earned in all courses 
by students in an analysis divided by the total number of credit hours attempted in all courses. 
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This measure accounts for course withdrawals and incompletes, as well as academic 
performance. 

Percentage of courses completed with a grade of “C” or better. The total number of 
courses in which a grade of “C” or better was earned by students in an analysis was divided by 
the total number of courses these students attempted. 

Grade performance in developmental courses. Average grade performance for students 
in an analysis for all developmental courses in which these students enrolled in reading, writing, 
or mathematics. 

Grade performance in gatekeeper courses. The average grade for students in common 
English and Mathematics “gatekeeper” courses that are required of all students in order to 
complete their academic programs. Specific “gatekeeper” courses identified by the Florida 
Community College System include English 1101, Math 1 033, and Math 1 1 05. 

Enrollment. The cumulative number of terms enrolled. 

Cumulative credits completed. The cumulative number of credits completed per student. 

Most of these performance measures were created for Long cohort files, with subsets 
calculated as appropriate for Short cohort files and the Cross-sectional performance file. 

Analyses 

Within each of the three analytical files (Long cohorts, Short cohorts, and the Cross- 
sectional performance file), three basic analytical methods were used to examine relationships 
between CCSSE benchmarks, item clusters, and the defined performance measures: bivariate 
correlations, regression analysis, and logistic regression analysis. For the cohort files, 
independent analyses were first performed for each starting cohort individually to determine if 
there were systematic differences in the relationships among variables over time or between 
students beginning their studies in the spring term as opposed to the fall term. No such 
differences were detected, so all individual cohort files were merged in order to assemble a large 
number of cases for analysis. 

Bivariate correlations were calculated for each possible pair of CCSSE benchmarks and 
item clusters and performance measures. This analysis examines CCSSE as a direct predictor of 
academic outcomes and behavior — i.e. a “proxy” for academic performance itself. These 
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analyses were not limited by missing data except any missing data that might be present in either 
of the two paired variables. 

Regression analyses were performed to determine the net effect of each CCSSE 
benchmark or item cluster on each performance measure. Control variables in the regression 
included gender, a dummy variable representing black, Hispanic, or Native American status, age 
at entry, number of years since high school completion at entry, placement test (CPT) scores in 
reading, writing, and math, and credit hour load. For those performance measures typically 
taking more time to complete — for example, degree completion or achievement of transfer-ready 
status — the cohort was also used as a control. The cumulative effects of missing data (principally 
CPT placement test scores) meant that these analyses generally were based on about one third 
fewer students in each file than the correlation analyses. 

Logistic regression models were constructed with controls identical to those used in the 
regression models and were used for binary performance measures (e.g., earning an associate 
degree or attaining “college path” status). Results for the OLS and logistic regressions were for 
the most part consistent, though a few differences were detected. 

In all of the regressions, student ability (as measured by CPT scores), selected 
demographics such as race/ethnic status, and identified risk factors were powerfully related to 
outcomes, leaving little additional variance for CCSSE constructs to account for. Under these 
conditions, the emergence of any significant effects for CCSSE benchmarks and item clusters 
indicates the presence of a net effect. Except for whether or not the CCSSE benchmark or item 
cluster emerged as a significant predictor, the strength and direction of relationships between 
performance measures and control variables in these regressions differed little across analyses. 
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Results 



Merged cohort results 

Merged cohort files were constructed on the basis of students beginning their studies at a 
Florida community college in fall or spring terms from fall 1 996 through fall 2002, with records 
updated through summer 2005. After all exclusions were applied, the working data file contained 
a total of 1958 cases. Because of the long period over which students were tracked, analyses of 
Long cohorts could examine a wide range of student outcomes, as indicated in Table 2. See 
Appendix Tables A1 - A1 0 for complete Merged Cohort results. 



Table 2 

Descriptive Statistics for Outcomes in Merged Cohorts 



Performance Measure 


Summary Results 


Earned LT Associate (%) 


2.9 


Earned Associate (%) 


37.1 


Earned Associate in 3 Years (%) 


21.7 


Took Gatekeeper Course (%) 


87.4 


Passed Gatekeeper Course (%) 


82.5 


Failed Gatekeeper Course (%) 


36.3 


Took Developmental Course (%) 


58.5 


Passed Developmental Course (%) 


53.0 


Failed Developmental Course (%) 


34.0 


Transfer-ready (%) 


23.9 


Enrolled Next Term (%) 


82.4 


Enrolled Next Year (%) 


76.9 


College Path by Next Term (%) 


64.1 


Overall GPA 


3.01 


Credit Completion Ratio 


80.7 


Classes Completed with C or Better (%) 


76.2 



CCSSE benchmarks and item clusters are significant bivariate and net predictors of 
college-level GPA, but are somewhat less well associated with credit-completion ratios and the 
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completion of courses with a grade of “C” or better after controls are introduced. With regard to 
GPA, all of the “academic” CCSSE benchmarks and item clusters are significantly associated 
with performance. All three outcome measures show significant net effects with the CCSSE item 
on Academic Gains, providing useful validation for this self report. 

CCSSE constructs are also significant bivariate and net predictors of overall associate 
degree completion, as well as degree completion within three years. Interestingly, Student- 
Faculty Interaction, along with Class Assignments and Exposure to Diversity item clusters, is not 
associated with degree completion. 

CCSSE constructs exhibit positive net effects on achieving transfer-ready status. 
Transfer-ready, it should be emphasized, is the most “academic” of the performance measures 
used, with the exception of GPA, so it is particularly interesting that it emerges as one of the 
stronger sets of net relationships with the CCSSE “academic” benchmark Academic Challenge, 
and the Academic Preparation and Mental Activities item clusters. Support for the validity of 
CCSSEs self-reported Gains in Academics item cluster is again provided by the emergence of 
this CCSSE item as a significant predictor. 

In contrast, fewer CCSSE benchmarks and item clusters are significantly related to early 
persistence — either to the next term or to the next year — after controls are introduced. But those 
net effects that emerged as significant are for item clusters that the literature suggests should be 
related to persistence — that is, Collaborative Learning and Student Services item clusters. 

CCSSE constructs have relatively weak relationships with taking and passing either 
developmental or gatekeeper courses — both direct and after controls. 

Short cohort Results 

Short cohort files were constructed on the basis of students beginning their studies at a 
Florida community college in fall or spring terms from fall 1 996 through fall 2004, with records 
updated for their first three terms of potential enrollment. After all exclusions were applied, the 
working data file contained a total of 2,658 cases. Because of the limited period over which 
students were tracked, analyses of Short cohorts could examine only a subset of the outcomes 
possible using Merged Cohorts, but with a greater number of cases. Summary results for 
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performance outcomes are summarized in Table 3. See Appendix Tables All - A1 7 for 
complete Short cohort results. 

Table 3 

Descriptive Statistics for Outcomes in Short cohorts 



Performance Measure 


Summary Results 


Enrolled Next Term (%) 


76.7 


College Path by Next Term (%) 


69.4 


Overall GPA 


2.84 


Credit Completion Ratio 


78.1 


Classes Completed with C or Better (%) 


81.7 


Took Gatekeeper Course (%) 


63.3 


Passed Gatekeeper Course (%) 


54.5 


Failed Gatekeeper Course (%) 


19.3 


Took Developmental Course (%) 


51.4 


Passed Developmental Course (%) 


57.1 


Failed Developmental Course (%) 


24.0 



Significant net effects on GPA within the first three terms of enrollment emerged only for 
Active and Collaborative Learning, Student Effort, and Class Assignments benchmarks while the 
validity of self-reported Academic Gains was again modestly confirmed. This suggests that the 
net effects of engagement on academic outcomes are more marked in later terms of enrollment — 
after a student has achieved “college path” status — than in the first three terms of enrollment. 2 
For credit completion, moreover, the Support for Learners benchmarks and the Class 
Assignments and School Opinions item clusters emerged as a significant net predictor within the 
first three terms of engagement. For the proportion of courses completed with a grade of “C” or 
better in the first three terms of enrollment, Academic Challenge and Academic Preparation 
showed significant net effects. Finally, self-reported Gain in Academics was again validated 
against a real measure of academic success. 



2 Short cohorts also had significantly lower GPA than Long cohorts (2.84 vs. 3.01) reflecting both 
the superior academic performance for “survivors” and the typical phenomenon at most 
institutions of increasing grades in later terms of enrollment for successful students. 
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Significant net effects on persistence to the next term emerged for a number of CCSSE 



constructs, including the Active and Collaborative Learning, Support for Learners, Student- 
Faculty Interaction benchmarks and the Collaborative Learning item cluster, while virtually all 
CCSSE constructs showed significant bivariate correlations. A somewhat stronger pattern of 
association — both bivariate and net — emerged for achieving “college path” status by the end of 
the first year of enrollment. 

Only a few significant net effects emerged for course performance in the Short cohort 
group. For developmental coursework, only the Active and Collaborative Learning benchmark, 
and the Class Assignments and Academic Preparation item clusters showed significant net 
effects, while for gatekeeper course performance, only Class Assignments showed a significant 
net effect. 

Cross-sectional performance file for Academic Year 2003-2004 

This file was constructed to correspond as closely as possible to the “academic year” to 
which students would be expected to be referring when they reported experiences and behaviors 
on the CCSSE in the spring of 2004. It contains all students who completed the CCSSE at that 
time and enrolled at any point in the fall 2003, spring 2004, or summer 2004 terms and records all 
academic activity within that time period. After all exclusions were applied, the working data file 
contained a total of 3,544 cases. Because this file was cross-sectional, persistence could not be 
investigated. And again, the limited period over which students were tracked, analyses could 
examine only a subset of the outcomes possible using Long cohorts, but with a greater number of 
cases. Summary results for performance outcomes are summarized in Table 4. 3 See Appendix 
Tables A18 - A21 for complete Cross-sectional performance file results. 

CCSSE constructs are significant bivariate and net predictors of college-level GPA, three- 
term credit completion ratios, and the percent of courses in which a grade of A through C was 
earned. All the CCSSE “academic” item constructs are related to all three of these outcomes, 
with Collaborative Learning and Student Services item clusters also significant net predictors for 



3 Note: Too few of the students in this dataset enrolled in developmental classes to support 
meaningful analyses. 
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credit completion ratio. All three measures show significant net effects with the Academic Gain 
item cluster, providing useful validation for this self-report. 

Table 4 

Descriptive Statistics for Outcomes in Cross-sectional performance file for Academic Year 2003- 
2004 



Performance Measure 


Summary Results 


Overall GPA 


2.89 


GPA in Gatekeeper Course 


2.53 


Credit Completion Ratio 


82.2 


Classes Completed with C or Better (%) 


80.6 



Only 27.2% of those included in the analysis took a gatekeeper course in the 2003-2004 
academic years, but grade-point performance for those who did take such courses shows 
significant bivariate and net effects for most CCSSE “academic” constructs, including the Student 
Effort and Academic Challenge benchmarks and the Class Assignments and Academic 
Preparation item clusters. 

Conditional Effects 

Because conditional effects have appeared intermittently in previous studies examining 
the relationship between CCSSE and NSSE responses and outcomes, a particular effort was 
made in this study to look for such effects in two areas: student academic ability and minority 
status. At issue was whether engagement matters more or less for students who enter with 
differing levels of academic ability or for minority students vs. white students. To investigate these 
questions, two sets of interaction variables were computed for all CCSSE item clusters in each of 
the three analytical files by multiplying each CCSSE benchmark or item cluster by total CPT 
score and by each race/ethnicity category. Each of these interaction variables was then entered 
into the regression on academic performance measures, together with all previous controls and 
the CCSSE construct to which the interaction variable corresponded. 

A number of significant interaction effects were revealed for entering ability in the 
analyses of all three datasets — Long cohorts, Short cohorts, and the 2003-2004 Cross-sectional 
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performance file. Consistent with previous studies of four-year institutions (Kuh, Kinzie, Cruce, 
Shoup, & Gonyea 2006; Cruce, Wolniak, Seifert, & Pascarella 2006), these showed that higher 
levels of engagement boosted GPA for students with low CPT scores, but not for students with 
high CPT scores. A graphic illustration of two of these conditional effects detected in the cohort 
files is provided in Figure 3 and Figure 4. 

Figure 3 

Conditional Effects of Academic Preparation and College Placement Tests on GPA 



Effect of "Academic Preparation" on GPA 
by Total CPT Group [Long Cohorts] 




0.2 0.4 0.6 0.8 1.0 

Engagement Level 



CPT 100 CPT 150 CPT 200 CPT 250 CPT 300 

1 
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Figure 4 

Conditional Effects of Class Assignments and College Placement Tests on GPA 



Effect of "Class Assignments" on GPA 
by Total CPT Group [Short Cohorts] 




Engagement Level 

CPT 100 CPT 150 CPT 200 CPT 250 CPT 300 

As is apparent, the regression model predicts that students in the lowest CPT ability 
groups in both cases gain markedly in GPA as their levels of engagement go up, while those in 
the highest ability group benefit less from engagement with respect to GPA, and their 
performance may even go down. 

Similar effects can be striking for students in the Cross-sectional performance file for 
Academic Year 2003-2004 — arguably, the dataset most suited to detecting the impact of 
engagement because academic outcomes were measured for the same year CCSSE responses 
were collected. The examples in Figure 5 and Figure 6 plot results for GPA and for the Credit 
Hour Completion Ratio. 
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Figure 5 

Conditional Effects of Academic Challenge and College Placement Tests on Three-Term GPA 



Effect of "Academic Challenge" on GPA 
by Total CPT Group [AY 2003-2004] 
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These cases are particularly interesting because students in the lowest ability group at 
the highest level of engagement rise to the performance levels attained by students in the highest 
ability group at the highest levels of engagement. 

Interaction effects of this kind between CCSSE constructs and CPT scores were found 
frequently for course-level performance. The most prominent among these were for the Support 
for Learners benchmark and the Class Assignments and Academic Preparation item clusters on 
GPA and Credit Completion Ratio. For less immediately academic outcomes like persistence 
and the achievement of “college path” status, similar conditional effects were found for the 
Support for Learners benchmark and the Student Services item cluster. Table 5, Table 6, and 
Table 7 note all instances where ability-related conditional effects emerged as significant at or 
below the .05 confidence level. 
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Figure 6 

Conditional Effects of Academic Challenge and College Placement Tests on Credit Completion 
Ratio 



Effect of "Student Effort" on Credit Completion Ratio 
by Total CPT Group [AY 2003-2004] 
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Table 5 



Significant Ability-Related Conditional Effects for GPA 









2003- 

2004 




Merged 


Short 


Acad. 


CCSSE Construct 


Cohorts 


cohorts 


Year 


Active and Collaborative Learning 


<.1 Level 






Student Effort 
Academic Challenge 
Student-Faculty Interaction 
Support for Learners 


<.1 Level 


Yes 


<.1 Level 


Faculty Interactions 
Class Assignments 


Yes 


Yes 




Exposure to Diversity 
Collaborative Learning 
Information Technology 
Mental Activities 


Yes 


<.1 Level 


<.1 Level 


School Opinions 
Student Services 




<.1 Level 




Academic Preparation 


Yes 


<.1 Level 





Table 6 

Significant Ability-Related Conditional Effects for Credit Completion Ratio 



2003- 

2004 



CCSSE Construct 


Merged 

Cohorts 


Short 

cohorts 


Acad. 

Year 


Active and Collaborative Learning 


Yes 




<.1 Level 


Student Effort 


Yes 




Yes 


Academic Challenge 






Yes 


Student-Faculty Interaction 






Yes 


Support for Learners 




Yes 




Faculty Interactions 






Yes 


Class Assignments 


Yes 


<.1 Level 


Yes 


Exposure to Diversity 








Collaborative Learning 


Yes 






Information Technology 








Mental Activities 






Yes 


School Opinions 




Yes 




Student Services 




<.1 Level 


<.1 Level 


Academic Preparation 


Yes 




<.1 Level 
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Table 7 



Significant Ability-Related Conditional Effects for Courses with A-C Grades 









2003- 








2004 




Merged 


Short 


Acad. 


CCSSE Construct 


Cohorts 


cohorts 


Year 


Active and Collaborative Learning 
Student Effort 


<.1 Level 




<.1 Level 


Academic Challenge 






Yes 


Student-Faculty Interaction 
Support for Learners 
Faculty Interactions 
Class Assignments 


Yes 


Yes 


<.1 Level 


Exposure to Diversity 
Collaborative Learning 
Information Technology 
Mental Activities 
School Opinions 
Student Services 
Academic Preparation 




Yes 


Yes 



Figure 7 

Conditional Effects of Active and Collaborative Learning and Race on Credit Completion Ratio 



Effect of "Active and Collaborative Learning" on Credit 
Completion Rate [AY 2003-2004] 
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Far fewer conditional effects of this kind were found for race/ethnicity, and the few that 
were identified were not always compensatory. That is, in some cases, greater levels of 
engagement as reflected in CCSSE responses benefited blacks and Hispanics more than they 
did whites in terms of academic outcomes, while in some cases the reverse was true. No 
conditional effects on race/ethnicity were found for less immediately academic outcomes like 
persistence and degree completion. Graphic illustrations of two typical, but opposite, conditional 
effects of this kind are displayed in Figure 7 and Figure 8. 

Figure 8 

Conditional Effects of Support for Learners and Race on GPA 
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In the case in Figure 7, both blacks and Hispanics gain markedly in credit-completion 
rates as their reported participation in behaviors associated with Active and Collaborative 
Learning increases; white students gain as well, but not so markedly. In the case in Figure 8, 
however, GPA is essentially unchanged for whites and blacks as Support for Learners increases, 
but decreases somewhat for Hispanic students. Table 8, Table 9, and Table 1 0 note all instances 
where interaction variables were significant at the .05 confidence level or beyond for African- 




American and Hispanic students and also indicate the direction of these conditional effects. 
30 



Table 8 



Significant Race-Related Conditional Effects for GPA 





African-American 




Hispanic 






Merged 


Short 


AY 


Merged 


Short 


AY 


CCSSE Construct 


Cohorts 


Cohorts 


2003-04 


Cohorts 


Cohorts 


2003-04 


Active and Collaborative Learning 
Student Effort 
Academic Challenge 
Student-Faculty Interaction 
Support for Learners 


Negative 


Positive 


Negative 


Negative 




Negative 


Faculty Interactions 
Class Assignments 
Exposure to Diversity 
Collaborative Learning 






Negative 






Positive 


Information Technology 
Mental Activities 






Negative 






Positive 


School Opinions 
Student Services 
Academic Preparation 


Negative 




Negative 


Negative 




Negative 



Table 9 

Significant Race-Related Conditional Effects for Credit Completion Ratio 





African-American 




Hispanic 










AY 










Merged 


Short 


2003- 


Merged 


Short 


AY 


CCSSE Construct 


Cohorts 


Cohorts 


04 


Cohorts 


Cohorts 


2003-04 


Active and Collaborative Learning 
Student Effort 






Positive 






Positive 


Academic Challenge 
Student-Faculty Interaction 
Support for Learners 






Positive 








Faculty Interactions 
Class Assignments 
Exposure to Diversity 
Collaborative Learning 
Information Technology 
Mental Activities 






Positive 








School Opinions 
Student Services 
Academic Preparation 


Positive 




Positive 






Negative 
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Table 10 



Significant Race-Related Conditional Effects for Courses with A-C Grades 





African-American 




Hispanic 






Merged 


Short 


AY 


Merged 


Short 


AY 


CCSSE Construct 


Cohorts 


Cohorts 


2003-04 


Cohorts 


Cohorts 


2003-04 


Active and Collaborative Learning 
Student Effort 
Academic Challenge 
Student-Faculty Interaction 
Support for Learners 






Negative 


Positive 




Negative 


Faculty Interactions 
Class Assignments 
Exposure to Diversity 
Collaborative Learning 






Negative 






Positive 


Information Technology 
Mental Activities 






Negative 








School Opinions 
Student Services 
Academic Preparation 


Positive 




Negative 


Positive 




Negative 



While patterns of results here are mixed, positive effects for African-Americans and 
Hispanics appear more likely to emerge in credit-completion than in graded academic 
performance and tend to be more associated with less “academic” CCSSE constructs, such as 
the Support for Learners benchmark and the School Opinions and Student Services item clusters. 
With regard to pure academic performance as reflected in GPA and percentage of courses with 
grades of C or better, negative effects strongly outnumber positive compensatory effects for 
African-American and Hispanic students. 



Discussion 

Overall, this pattern of results broadly confirms the presence of positive relationships 
between the construct of student engagement as measured by CCSSE and community college 
student outcomes. CCSSE benchmarks and item clusters show a consistent pattern of significant 
association with academic outcomes like GPA, degree completion, and attaining important 
academic milestones like “college path” and “transfer-ready” status after controlling for student 
characteristics and entering ability. And the strongest of these net effects materialize where they 
are most expected — for “academic” areas of engagement such as Academic Challenge, Active 
and Collaborative Learning, Student-Faculty Interaction, and Mental Activities. At the same time, 
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self-reported Academic Gains on CCSSE are significantly related to actual academic 
achievement measures like GPA, achieving “transfer-ready” status, and degree completion, both 
directly (confirmed through bivariate correlation analysis) and after controlling for student ability 
and background. 

While pervasive significant net effects of this kind are less typical of behavioral measures 
of student success like persistence to a second term or year, they do occur repeatedly across 
both longitudinal cohort datasets. Moreover, the CCSSE constructs that emerge as significant in 
these cases are those that the retention literature says should do so: Support for Learners and 
Student Services (and occasionally Collaborative Learning). 

Two CCSSE item clusters do not appear to influence outcomes of either kind: Exposure 
to Diversity and Information Technology. This is consistent with much previous work on CCSSE. 

The emergence of conditional effects, though less pervasive than direct effects, confirms 
the results of similar studies using NSSE and other four-year academic outcomes (Kuh et. al. 
2006; Cruce et. al. 2006; Carini, Kuh, & Klein 2006) about the compensatory value of 
engagement for lower-ability students. And these interaction effects are also in expected 
directions — academic factors related to academic outcomes like GPA and more supportive 
factors related to behavioral outcomes like persistence. But the conditional effects uncovered for 
race/ethnicity are mixed, with some evidence of compensatory effects for African-Americans and 
Hispanics emerging for less “academic” forms of engagement on credit completion ratios, but 
generally negative outcomes for pure academic performance. 

Finally, one caveat that must be placed on these results is the fact that the study sample 
is skewed toward “traditional” community college students. While exploratory analyses revealed 
no significant differences in these patterns of association between younger and older students, 
full-time enrollees vs. part-time enrollees, or AA-seekers versus students seeking credentials 
other than the AA, there were too few cases of non-traditional students in the core sample to 
allow such differences to be entirely ruled out. 
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Study 2: Achieving the Dream Validation Study 

Study Description 

Sample Overview 

Data from 24 community colleges in the Achieving the Dream initiative were analyzed. 
These data were merged with CCSSE survey data to examine the relationship between student 
engagement and perceived Gains based on CCSSE responses and student outcome information 
from administrative data reported by colleges for Achieving the Dream. There were a total of 
1,623 students who completed the CCSR in a primary CCSSE sample and provided an ID that 
could be matched to a record in the Achieving the Dream database. 

Approximately 95% of the sample responded to CCSSE in either the 2004 or 2005 
administrations. The sample was split among each of the three Achieving the Dream cohorts: 
31% began in 2002, 44% began in 2003, and 24% began in 2004. Thus, almost 75 percent of the 
sample had at least four terms of data (excluding the summer terms which were sparsely 
populated in the Achieving the Dream database), and the entire sample had at least one 
academic term of data (fall and spring). All analyses were conducted on the complete sample 
across all cohorts unless otherwise stated. Table 1 1 illustrates frequency characteristics for key 
control variables used in the validation study: gender, race and ethnicity, part-time status, and 
age. 

These data indicate that the merged sample is much younger than the overall Achieving 
the Dream universe, and more likely to be women. Students in the merged database are much 
less likely to enroll part-time (28% vs. 40%) in their first term. Although the merged sample has a 
slightly lower proportion of blacks (non-Hispanics) (14.0% vs. 1 6.6%) than the overall Achieving 
the Dream universe and a slightly higher proportion of Hispanics (34.8% vs. 30.8%), the race and 
ethnic distribution of the merged sample remains predominantly non-white (59%) - which reflects 
the college eligibility requirement of the Achieving the Dream initiative. 
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Table 1 1 



Race, Ethnic, Gender, Age and Part-Time Status: Achieving the Dream Universe and Merged 
Analytic Sample 



Variable 


Achieving the 
Dream Universe 


Merged Analytic Sample 


Gender (%) 


Male 


43.4 


35.3 


Female 


55.2 


64.7 


Race and Ethnicity (%) 


Black, non-Hispanic 


16.6 


14.0 


White, non-Hispanic 


40.3 


41.8 


Hispanic 


30.4 


34.8 


Other 


12.7 


9.4 


Part-Time Status (Year 1, Term 1) (%) 


Yes 


40.8 


28.3 


No 


59.2 


71.7 


Age (%) 


25 or older 


28.1 


22.9 


24 or younger 


71.9 


77.1 


N 


244,675 


1,623 



The merged analytic sample indicates the significant need for students at Achieving the 
Dream colleges to enroll in developmental education courses: almost two-thirds (63%) placed at 
least one-level below college math, about one-third (33%) placed at least one-level below college 
English, and 35 percent placed at least one-level below college reading. 



Data Construction 

The first step of the validation study involved merging a database of community college 
students at Achieving the Dream institutions who began in 2002, 2003 or 2004 - the final analysis 
used the Achieving the Dream database from July 6, 2006 - with a CCSSE database of students 
who took the CCSSE at one of the Achieving the Dream colleges between 2002 and 2005. There 
were 5,551 students who provided student IDs. Of these 5,551 students, 1 ,623 CCSSE 
respondents voluntarily provided a unique student identifier that allowed their responses to be 
matched with Achieving the Dream administrative records. 
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Study Variables 

The Achieving the Dream database includes developmental education, college algebra, 
and college English course information, enrollment data for each term, and degree or certificate 
attainment information. The administrative records also include basic student demographics, 
including gender, race and ethnicity, and age. The CCSSE database provides information on 
CCSSE benchmarks, engagement item clusters, and perceived Gains item clusters. Additionally, 
an indicator of risk factors constructed from CCSSE response data was derived. Several 
outcome variables were created for the validation study and are described below. 

Enrollment. For each term in the database, a variable measuring cumulative fall and 
spring terms enrolled was created. 

College algebra course completions. For each term in the database, binary variables 
were constructed for students who completed College Algebra with a ‘C’ or better. Additionally, a 
binary variable was constructed for completion of College Algebra at any time up to the third year 
spring term. 

College English course completions. For each term in the database, binary variables 
were constructed for students who completed College English with a ‘C’ or better. Additionally, a 
binary variable was constructed for completion of College English at any time up to the third year 
spring term. 

Developmental math course completions. For each term in the database, binary variables 
were constructed for students who completed a developmental math course - by level - with a ‘B’ 
or better. Additionally, binary variables were constructed for completion of developmental math - 
by level - at any time up to the third year spring term. 

Developmental English course completions. For each term in the database, binary 
variables were constructed for students who completed a developmental English course - by level 
- with a ‘B’ or better. Additionally, binary variables were constructed for completion of 
developmental English - by level - at any time up to the third year spring term. 

Developmental reading course completions. For each term in the database, binary 
variables were constructed for students who completed a developmental reading course - by level 
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- with a ‘B’ or better. Additionally, binary variables were constructed for completion of 
developmental reading - by level - at any time up to the third year spring term. 

Cumulative GPA. For each term in the database, cumulative grade point average is 
reported. 

Cumulative credits completed. For each term in the database, a variable measuring 
cumulative credits completed from the first through the third year was created. 

Credit-completion ratios. For each term in the database, credit completion ratios were 
constructed as a measure of the number of credits completed divided by the number of credits 
attempted. In addition, a cumulative ratio variable was constructed as a measure of the number of 
credits completed divided by the number of credits attempted for the first and second year, and 
for the first through third year. 

Persistence. For each term in the database, an enrollment flag was created to account 
for students who attempted both credit and non-credit courses. An intermediate persistence 
variable was derived using these enrollment flags: year-to-year persistence from fall to fall, year 1 
to year 2. 

Attainment. Degree or certificate completion flags were created for all students in the 
merged analytic database. 

Analyses 

Three basic methods were used to examine these relationships. First, an equality of 
means test (ftest) was used to examine differences in CCSSE benchmarks between different 
groups of students. Second, bivariate correlations were calculated for each possible pair of 
CCSSE constructs and Achieving the Dream outcome variables. Finally, each of these 
relationships was further examined through regression analyses to estimate the net effect of each 
CCSSE benchmark, engagement item cluster, and perceived Gains item cluster on each 
outcome measure (logistic regression was used for binary dependent variables, and linear 
regression was used for continuous dependent variables). Control variables in the regression 
included gender, race and ethnicity, age, developmental math placement levels, part-time status, 
and a risk index created from CCSSE responses. In addition, the Achieving the Dream cohort 
was used as control. In all, 17 regressions were run for each outcome measure. 
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Results 



Comparison of CCSSE Benchmark Means 

Race and ethnicity. Table 1 2 illustrates the mean CCSSE benchmarks for different 
groups of students according to race and ethnic characteristics, nests were conducted between 
black and white students, and between Hispanic and white students; results of this statistical test 
indicates that black students are more engaged than white students on the Student Effort, 
Academic Challenge, and Support for Learners benchmarks; however, there were no statistical 
differences in mean benchmark scores for Active and Collaborative Learning or Student-Faculty 
Interaction. Hispanic students are more engaged than white students on the Student Effort and 
Support for Learners benchmarks, but no statistical differences were found for Active and 
Collaborative Learning, Student-Faculty Interaction, and Academic Challenge. 



Table 12 

Comparison of CCSSE Benchmark Means by Race and Ethnicity 



CCSSE Benchmark 


Mean 


t 


Sig. 


N 


Active and Collaborative Learning 










Black, non-Hispanic 


.4018 


1.598 


.110 


227 


Hispanic 


.3789 


-.283 


.776 


565 


White, non-Hispanic 


.3815 






678 


Student Effort 










Black, non-Hispanic 


.5195 


4.096 


.000 


227 


Hispanic 


.4978 


3.224 


.001 


565 


White, non-Hispanic 


.4694 






905 


Academic Challenge 










Black, non-Hispanic 


.6175 


4.049 


.000 


227 


Hispanic 


.5706 


.645 


.519 


565 


White, non-Hispanic 


.5646 






905 


Student-Faculty Interaction 










Black, non-Hispanic 


.3875 


.224 


.823 


227 


Hispanic 


.3662 


-1.719 


.086 


565 


White, non-Hispanic 


.3843 






677 


Support for Learners 










Black, non-Hispanic 


.5218 


5.685 


.000 


224 


Hispanic 


.4924 


5.199 


.000 


565 


White, non-Hispanic 


.4307 






678 



NOTE: f tests for black and Hispanic are based on comparison with white 



Public assistance. Table 1 3 provides results for comparisons between students who 
reported that public assistance was a major source of support for college enrollment and those 
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who reported that public assistance was not a source of support. This survey item was used as a 
proxy for low-income status; this statistical test suggests that low-income students are more 
engaged than higher income students on four of five CCSSE benchmarks: Active and 
Collaborative Learning, Student Effort, Student-Faculty Interaction, and Support for Learners. 
Although the Achieving the Dream database has administrative records for Pell grant receipt (a 
typical proxy for low-income), these data are considered unreliable due to reporting problems 
from participating colleges and were not used for this analysis. 

Table 13 

Comparison of CCSSE Benchmark Means, by Income Proxy (Public Assistance as Source of 
Support) 



CCSSE Benchmark 


Mean 


t 


Sig. 


N 


Active and Collaborative Learning 










Public Assistance Major Source 


.4344 


3.998 


.000 


152 


Public Assistance Not a Source 


.3783 






1,309 


Student Effort 










Public Assistance Major Source 


.5219 


2.667 


.008 


152 


Public Assistance Not a Source 


.4864 






1,309 


Academic Challenge 










Public Assistance Major Source 


.5976 


1.702 


.089 


152 


Public Assistance Not a Source 


.5731 






1,309 


Student-Faculty Interaction 










Public Assistance Major Source 


.4203 


3.009 


.003 


152 


Public Assistance Not a Source 


.3732 






1,309 


Support for Learners 










Public Assistance Major Source 


.5587 


5.835 


.000 


152 


Public Assistance Not a Source 


.4520 






1,308 



NOTE: Public Assistance Minor Source is not included in the table 



Grants and scholarships. An alternative measure of students’ economic background is 
CCSSE responses to reliance on grants and scholarships to pay for college. Table 14 provides 
the results of a statistical test comparing students who replied grants and scholarships were a 
major source of financial support with students who replied that gift aid was not a source at all. 
Based on this measure, students from lower income backgrounds are more engaged than higher 
income students on all CCSSE benchmarks. 
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Table 14 



Comparison of CCSSE Benchmark Means, by Income Proxy (Grants and Scholarships as Source 
of Support) 



CCSSE Benchmark 


Mean 


t 


Sig. 


N 


Active and Collaborative Learning 










Grants & Scholarships Major Source 


.3962 


2.814 


.005 


697 


Grants & Scholarships Not a Source 


.3721 






728 


Student Effort 










Grants & Scholarships Major Source 


.5145 


6.184 


.000 


697 


Grants & Scholarships Not a Source 


.4646 






728 


Academic Challenge 










Grants & Scholarships Major Source 


.5959 


4.063 


.000 


697 


Grants & Scholarships Not a Source 


.5600 






728 


Student-Faculty Interaction 










Grants & Scholarships Major Source 


.3931 


3.019 


.003 


697 


Grants & Scholarships Not a Source 


.3637 






728 


Support for Learners 










Grants & Scholarships Major Source 


.5157 


8.079 


.000 


696 


Grants & Scholarships Not a Source 


.4247 






728 



NOTE: Grants & Scholarships Minor Source is not included in the table 



Risk factors. Table 15 provides the results for a fourth comparison of means - between 
students with two or more risk factors and students with zero risk factors. Risk factors were 
defined by CCSSE staff and include part-time enrollment status, need for developmental 
education, single parents, students who work more than 30 hours weekly, first-generation 
students, and a financial flag indicating that financial issues are very likely to cause withdrawal 
from college. In the merged analytic file, less than 6 percent of students had zero risk factors and 
more than 70 percent had two or more. The statistical test indicates that students with two or 
more risk factors are more engaged than students with zero risk factors on only one CCSSE 
benchmark: Student Effort. 
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Table 15 



Comparison of CCSSE Benchmark Means, by Student Risk Factors 



CCSSE Benchmark 


Mean 


t 


Sig. 


N 


Active and Collaborative Learning 










Two or more risk factors 


.3840 


-.185 


.853 


1171 


Zero risk factors 


.3871 






103 


Student Effort 










Two or more risk factors 


.4994 


3.412 


.001 


1171 


Zero risk factors 


.4458 






103 


Academic Challenge 










Two or more risk factors 


.5803 


1.120 


.263 


1171 


Zero risk factors 


.5608 






103 


Student-Faculty Interaction 










Two or more risk factors 


.3784 


.346 


.730 


1171 


Zero risk factors 


.3718 






102 


Support for Learners 










Two or more risk factors 


.4764 


.837 


.403 


1171 


Zero risk factors 


.4573 






101 



NOTE: Students with only one risk factor are not included in the table 



Bivariate Correlations and Net Effects 

Bivariate correlations were calculated for each possible pair of CCSSE constructs and 
Achieving the Dream outcome variables. Each of these relationships was further examined 
through regression analyses to estimate the net effect of each CCSSE construct. See Appendix 
Tables B1 - B17 for complete correlation and regression results 

Gatekeeper courses. One CCSSE benchmark - Active and Collaborative Learning - is 
positively related to completion of College Algebra with a ‘C’ or better. In addition, three CCSSE 
item clusters and one gain measure had statistically significant bivariate correlations. Moreover, 
logistic regression results indicate positive net effects for the Active and Collaborative Learning 
benchmark, and for the Class Assignments and Collaborative Learning item clusters. Students’ 
perception of Academic Gains also had positive net effects when predicting completion of College 
Algebra with a ‘C’ or better by the third year. 

The results for completion of College English with a ‘C’ or better by the third year were 
less promising - none of the CCSSE benchmarks had statistically significant bivariate 
correlations, and only one item cluster - Information Technology - had a statistically significant 
bivariate correlation. There were no net effects of CCSSE benchmarks, item clusters, or 
perceived Gains when predicting completion of College English by the third year. 
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Developmental education. Two CCSSE benchmarks - Student Effort and Academic 
Challenge - had statistically significant bivariate correlations with the completion of 
developmental mathematics - one level below college - with a ‘B’ or better; these correlations did 
not hold in the regression analyses. Two item clusters - Academic Preparation and Faculty 
Interactions - also had statistically significant bivariate correlations. Students’ perceptions of 
Academic Gains had both a statistically significant correlation, and a positive “net effect” when 
predicting completion of developmental math level 1 by the third year. 

For completion of developmental math - two levels below college - students’ School 
Opinions and Academic Preparation had statistically significant bivariate correlations with 
completion of developmental math level 2 with a ‘B’ or better by the third year. Additionally, 
students’ perceived Academic Gains were also statistically significant. The perceived Academic 
Gains measure and the Academic Preparation item cluster also had positive net effects when 
predicting completion of developmental math level 2. No CCSSE benchmarks had statistically 
significant relationships with the completion of developmental math level 2. 

Only two measures - the Active and Collaborative Learning benchmark and the 
Academic Preparation item cluster had statistically significant bivariate correlations with 
completion of developmental math - three levels below college - with a ‘B’ or better by the third 
year. Two additional item clusters - Class Assignments and Collaborative Learning - had positive 
net effects when predicting developmental math level 3 course completions; the Active and 
Collaborative Learning benchmark also had positive net effects when predicting the completion of 
developmental math level 3 with a ‘B’ or better. 

Results for developmental English were less promising - only the Academic Preparation 
item cluster had a statistically significant bivariate correlation with the completion of 
developmental English (writing) - one level below college - with a ‘B’ or better by the third year. 
This item cluster also had a statistically significant bivariate correlation with the completion of 
developmental English two or more levels below college. Students’ perceived Academic Gains 
were also statistically related to the completion of developmental English level 2 with a ‘B’ or 
better by the third year. There were no net effects for CCSSE benchmarks, item clusters and 
Gains when predicting the completion of developmental English two or more levels below college 
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by the third year. In contrast, two measures - students’ School Opinions and the Support for 
Learners benchmark - had negative net effects when predicting the completion of developmental 
English level 1 with a ‘B’ or better by the third year. 

Results for developmental reading were more positive than those for developmental 
English. One CCSSE benchmark - Student Effort - had a statistically significant bivariate 
correlation with the completion of both developmental reading one and two levels below college. 
One item cluster - Class Assignments - also had a statistically significant bivariate correlation 
with developmental reading level 1 and level 2. An additional item cluster - Information 
Technology - had a statistically significant bivariate correlation with completing developmental 
reading level 2 with a ‘B’ or better by the third year. Each of the measures with statistically 
significant bivariate correlations also had positive net effects when predicting the completion of 
developmental reading level 2 with a ‘B’ or better by the third year: the Student Effort benchmark, 
and the Class Assignments and Information Technology item clusters. One item cluster - Class 
Assignments - also had a positive “net effect” when predicting completion of developmental 
reading level 1 with a ‘B’ or better by the third year. 

Academic achievement. Bivariate correlations and positive net effects were present for 
CCSSE benchmarks, item clusters, and academic achievement; however, there were no 
statistical relationships between students’ perceived Gains and academic achievement. Three 
CCSSE benchmarks - Active and Collaborative Learning, Academic Challenge, and Student - 
Faculty Interaction had both statistically significant bivariate correlations and positive net effects 
when predicting cumulative grade point average after two years of college. Moreover, six item 
clusters - Faculty Interactions, Class Assignments, Exposure to Diversity, Collaborative Learning, 
Mental Activities, and Academic Preparation - had both statistically significant bivariate 
correlations and positive net effects when predicting cumulative GPA. 

Persistence. Two measures of persistence were used for the validation study: credit 
completion ratios (year 1 through year 2) and fall-to-fall retention, year 1 to year 2. Bivariate 
correlations and positive net effects were present for CCSSE benchmarks, engagement item 
clusters and perceived academic Gains item clusters, and credit completion ratios through year 2. 
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Four CCSSE benchmarks had statistically significant bivariate correlations and positive 
net effects when predicting cumulative credit completion ratios after two years: Active and 
Collaborative Learning, Student Effort, Academic Challenge and Student-Faculty Interaction. Five 
additional item clusters also had statistically significant bivariate correlations and positive net 
effects when predicting cumulative credit completion ratios: Faculty Interactions, Class 
Assignments, Information Technology, Mental Activities, and Academic Preparation. Students’ 
perceived Academic Gains also had a statistically significant bivariate correlation and positive 
“net effect” when predicting cumulative credit completion ratios after two years. 

Similar results - but with fewer measures - were found when using persistence from the 
fall of year 1 to the fall of year 2 as a persistence measure. One CCSSE benchmark - Active and 
Collaborative Learning - had a statistically significant bivariate correlation and positive “net effect” 
when predicting year-to-year persistence. Three item clusters - Collaborative Learning, 
Information Technology, and Student Services - also had statistically significant bivariate 
correlations and positive net effects when predicting fall to fall persistence. Students’ perceived 
Academic Gains was the final measure with statistically significant bivariate correlations and 
positive net effects when predicting persistence. 

Attainment. The results for degree or certificate attainment after three years were also 
positive. Three CCSSE benchmarks - Active and Collaborative Learning, Academic Challenge, 
and Student-Faculty Interaction had statistically significant bivariate correlations and positive net 
effects when predicting graduation. Additionally, three item clusters had statistically significant 
bivariate correlations and positive net effects when predicting degree or certificate attainment 
after three years: Faculty Interactions, Collaborative Learning, and Academic Preparation. 

Further, students’ perceived Career Gains has a statistically significant bivariate correlation and 
positive “net effect” when predicting graduation. 

Within-term effects for GPA and credit completion ratios. A more discrete validation 
analysis was conducted according to the academic year a student was administered the CCSSE; 
for Achieving the Dream colleges, 60 percent of students took the survey in the spring of their first 
year and 34 percent of students took the survey in the spring of their second year. 
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CCSSE benchmarks, engagement item clusters, and perceived academic Gains item 
clusters were correlated and had positive net effects when predicting same-term credit 
completion ratios and cumulative GPA for students who took the CCSSE in their first academic 
year. Four CCSSE benchmarks - Active and Collaborative Learning, Student Effort, Academic 
Challenge, and Student-Faculty Interaction — positively predicted credit completion ratios during 
the spring term of the respondents’ first academic year. Three of these benchmarks also 
predicted cumulative GPA: Active and Collaborative Learning, Student Effort, and Academic 
Challenge. Four engagement item clusters also had positive net effects when predicting same- 
term credit completion ratios and cumulative GPA: Faculty Interactions, Class Assignments, 
Mental Activities, and Academic Preparation. One additional item cluster - Information 
Technology - was also a positive predictor of same-term credit completion ratios. Finally, 
students’ perceived Academic Gains had a positive “net effect” when predicting credit completion 
ratios. 

The bivariate correlations and net effects for CCSSE benchmarks when predicting 
cumulative GPA after two years for students who took the CCSSE in their second academic year 
were also present; however, there were no net effects and only one bivariate correlation 
(Academic Preparation) with same-term credit completion ratios for students who took the 
CCSSE in the spring of their second academic year. Three CCSSE benchmarks - Active and 
Collaborative Learning, Student Effort, and Academic Challenge - and six item clusters (Faculty 
Interactions, Class Assignments, Exposure to Diversity, Collaborative Learning, Mental Activities 
and Academic Preparation) had positive net effects when predicting cumulative GPA after two 
years. In contrast, one CCSSE benchmark - Support for Learners - had a negative “net effect” 
when predicting cumulative GPA for students who took the CCSSE in the spring of their second 
academic year. Additionally, students perceived Career Gains were also a negative predictor of 
cumulative GPA. 
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Discussion 



The study examined overall differences in the levels of engagement between low-income 
students and other students, students of color and other students, and “high-risk” students and 
“low-risk” students. When using two CCSSE items as proxies for low-income status - reliance on 
grants and scholarships, and reliance on public assistance - there are statistical differences in 
mean CCSSE benchmark scores between low-income students and other students. Specifically, 
low-income students were more engaged than other students on at least four (and possibly all) of 
the CCSSE benchmarks: Active and Collaborative Learning, Student Effort, Student-Faculty 
Interaction, and Support for Learners. When using Achieving the Dream colleges’ administrative 
records, and identifying students by race and ethnic categories, there are statistical differences in 
mean CCSSE benchmark scores between students of color and other students. Black, non- 
Hispanic students were more engaged than white students on the Student Effort, Academic 
Challenge and Support for Learners benchmarks. Hispanic students were more engaged than 
white students on the Student Effort and Support for Learners benchmarks. A risk factor 
measure, the total number of risk factors a student possessed, revealed statistically significant 
differences in mean CCSSE benchmark scores between “high-risk” and “low-risk” students on 
only one benchmark: Student Effort. “High-risk” students were more engaged than “low-risk” 
students on this measure. 

The study examined whether engagement factors predict within-term persistence and 
whether engagement factors predict long-term persistence. Using credit completion ratios as a 
measure of within-term persistence indicates positive net effects for CCSSE benchmarks and 
item clusters when predicting credit completion ratios within the same term CCSSE was 
administered - if students took the CCSSE in the spring of their first year. The same measure for 
students who took the CCSSE in the spring of their second year yielded no “net effects.” Long- 
term persistence was measured two ways - cumulative credit completion ratios after two years, 
and fall-to-fall persistence year 1 to year 2. Four of the five CCSSE benchmarks - Active and 
Collaborative Learning, Student Effort, Academic Challenge, and Student-Faculty Interaction - 
had positive net effects when predicting cumulative credit completion ratios; several item clusters 
and students’ perceived Academic Gains were also positive predictors of credit completion ratios 
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after two years. Using fall-to-fall persistence as the outcome measure yielded positive net effects 
for the Active and Collaborative Learning benchmark as well as three CCSSE item clusters 
(Collaborative Learning, Information Technology, and use of Student Services) and students’ 
perceived Academic Gains. 

Completion of developmental and gatekeeper courses was examined to determine the 
extent to which engagement factors predict these outcomes. Using completion of developmental 
math, writing, and reading with a ‘B’ or better within three years yielded mixed results. For 
developmental math, the most promising results were at three levels below college; the Active 
and Collaborative Learning benchmark had a positive “net effect” when predicting course 
completion with a ‘B’ or better. No other benchmarks were positive predictors of developmental 
math course completions at any level. At the same time, students’ perceived Academic Gains had 
a positive “net effect” when predicting developmental math course completions-level 1 and level 
2-with a ‘B’ or better within three years. 

For developmental English (writing), engagement does not predict successful course 
completion with a ‘B’ or better two or more levels below college English; moreover, the Support 
for Learners benchmark is a negative predictor - that is, students with higher scores on this 
benchmark are less likely to complete developmental English one level below college with a ‘B’ or 
better within three years. 

For developmental reading, the most promising results were at two levels below college; 
the Student Effort benchmark had positive net effects when predicting developmental reading 
course completions with a ‘B’ or better within three years. The Class Assignments and 
Information Technology item clusters were also positive predictors of completing developmental 
reading level 2 with a ‘B’ or better. There were no net effects for CCSSE benchmarks when 
predicting course completions of developmental reading one level below college; however, the 
Class Assignments item cluster was a positive predictor of completing developmental reading 
level 1 with a ‘B’ or better within three years. 

Using completion of college Algebra and college English with a ‘C’ or better within three 
years yielded mixed results. The most promising results were for college Algebra: the Active and 
Collaborative Learning benchmark had a positive “net effect” when predicting the completion of 
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college Algebra within three years. Two item clusters (Class Assignments and Collaborative 
Learning) were also positive predictors of completing college Algebra with a ‘C’ or better, as was 
students’ perceived Academic Gains. There were no net effects for CCSSE benchmarks when 
predicting the completion of college English with a ‘C’ or better within three years. 

In addition to examining developmental course completion, the relationship between 
engagement and completion of developmental courses across all levels of developmental needs 
was examined. The relationship between engagement and completion of developmental courses 
varies across levels. In fact, CCSSE benchmarks tend to predict completion of developmental 
math and reading at the lowest levels measured in this report. For developmental math level 3, 
Active and Collaborative Learning is the key predictor of successful completion with a ‘B’ or 
better, while Student Effort is the key predictor of successful completion of developmental reading 
level 2 with a ‘B’ or better. Students’ perceived Academic Gains is a positive predictor of 
completing developmental math, levels 1 and 2; and the item cluster, Class Assignments, is a 
positive predictor for completing developmental reading, levels 1 and 2. 

The net effects for engagement when predicting degree or certificate attainment within 
three years were very positive. Three CCSSE benchmarks - Active and Collaborative Learning, 
Academic Challenge, and Student-Faculty Interaction - had positive net effects when predicting 
graduation. Three item clusters (Faculty Interactions, Collaborative Learning, and Academic 
Preparation) were also positive predictors of graduation, as were students’ perceived Career 
Gains. 

CCSSE benchmarks positively predict cumulative GPA after two years and cumulative 
GPA at the end of the term in which CCSSE was administered. Overall, Active and Collaborative 
Learning, Academic Challenge, and Student-Faculty Interaction had positive net effects when 
predicting cumulative GPA. Using cumulative GPA at the end of the term in which students took 
the CCSSE also yielded promising results: three benchmarks (Active and Collaborative Learning, 
Student Effort, and Academic Challenge) were positive predictors of cumulative GPA. Several 
CCSSE item clusters were also positive predictors of cumulative GPA; in contrast - for students 
who took the CCSSE during spring of their second year, perceived Career Gains was a negative 
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predictor of cumulative GPA. That is, students who believed they made larger career Gains had 
lower GPAs. 

All regression models included controls for race and ethnicity (binary variables for black, 
Hispanic and white); however, we did not have reliable measures for low-income status. In most 
cases, the control variables were not statistically significant in the regression models. In those 
cases where race and ethnicity did impact the predictive power of engagement, the effects were 
as expected: black and/or Hispanic students were less likely to have a successful outcome and 
white students were more likely to have a successful outcome. This impact was not widespread in 
these analyses, which suggests that engagement measures can predict student outcomes 
regardless of students’ race and ethnic characteristics. 
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Study 3: HSS Consortium Institutions Validation Study 

Study Description 

Sample Overview 

The CCSSE HSS consortium consists of community colleges that are either members of 
the Hispanic Association of Colleges and Universities (HACU) or have student populations 
comprised of greater than 25% Hispanic students. Approximately 27 percent (n = 3,540) of the 
16 reporting consortium college students in the CCSSE sample identified themselves as Hispanic 
and 23 percent of the sample indicated that English was not their first language (the vast majority 
of non-English fluency being Spanish). In fact, 48.9 percent of students who identified as Hispanic 
indicated that English was not their first language. Nine percent of the sample indicated they were 
born outside of the United States. 

There were a total of 3,279 students who completed the CCSR in a primary CCSSE 
sample and provided an ID and were thus included in the CCSSE validation sample. Of these, 
approximately 33 percent identified themselves as Hispanic and 26 percent indicated that English 
was not their first language. Forty-eight percent of Hispanics indicated that English was not their 
first language. Nine percent of these students indicated they were born outside the United States. 
Descriptive analyses (Table 1 6) showed that the group of students for whom we have transcript 
data was representative of students at participating institutions and of all students who completed 
the CCSSE at the 16 reporting consortium colleges. 

Data Construction 

The data from this study derived from three sources: the Community College Survey of 
Student Engagement {CCSSE), the National Center for Education Statistics (NCES) Integrated 
Postsecondary Data System (IPEDS), and the CCSSE HSS consortium participant institutions. 
Sixteen CCSSE HSS institutions provided data. The 16 institutions had 12,598 unweighted 
records with 3,509 “valid”’ IDs, and we obtained matches and transcript data for 2,778 student 
records (a 79.2 percent match rate). The weighted records, which are reported in all further 
analyses, yielded a total sample of 12,962 cases with 4,109 valid IDs and 3,279 matches (79.8 
percent match rate). 
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Table 16 



Study Community Colleges’ Demographic Comparison of IPEDS, CCSSE Sample, and CCSSE 
Validation Data 



Variable 


2004 Fall IPEDS 


CCSSE Sample 


CCSSE Validation 


Gender % 


Female 


60.1 


60.8 


62.1 


Male 


39.9 


39.2 


37.9 


Attendance % 


Full-Time 


29.3 


38.4 


44.0 


Part-Time 


70.7 


61.6 


56.0 


Race/Ethnicity % 


White, NH 


41.9 


43.6 


39.5 


Black, NH 


16.1 


11.0 


11.5 


Hispanic 


33.0 


27.3 


33.1 


Asian/PI 


4.2 


4.6 


5.3 


American Indian 


.5 


1.5 


1.4 


Other/Unknown 


4.2 


12.0 


9.3 


N 


265,689 


12,962 


3,279 



CCSSE data included all data elements from the 2002, 2003, and 2004 administrations of 
the survey. Select IPEDS data were downloaded from the NCES website to assess sample 
representativeness and to include institution-level variables in the analysis. CCSSE HSS 
consortium participants provided CCSSE staff with records from students who had completed the 
CCSSE in 2002-2004 and had provided valid SSNs. CCSSE staff summarized this data into an 
HSI dataset containing term data through spring 2005 for each student. These files were merged 
with the CCSSE and IPEDS data. 

Study Variables 

Cumulative GPA. For each term in the database, cumulative GPA was obtained. 

First to second term persistence. For each term in the database, an indicator of 
persistence from first to second term was created. 

First to third term persistence. For each term in the database, an indicator of persistence 
from first to third term was created. 
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Total credit hours taken. For each term in the database, the total number of credit hours 
taken was obtained. 

Enrollment. For each term in the database, a variable measuring cumulative enrollment 
terms was created. 

Average credit hours completed. For each term in the database, a measure of average 
credit hours completed per term was created. 

Table 17 

Description of HSS Study Variables 



Variables 


Source/Description 


Variable Type Per Question 


Satisfaction Items: 


Overall satisfaction 


CCSSE Item 


Q1, 2, &4DV Q3IV 


Likelihood of recommendation 


CCSSE Item 


Q1, 2, &4DV Q 


Institution-level: 


Size 


IPEDS Item 


Q2M Q3IV 


Urbanicity 


IPEDS Item 


Q2M Q3IV 


Graduation rate 


IPEDS Item 


Q2M Q3IV 


Proportion of part-time students 


IPEDS Item 


Q2M Q3IV 


Student-level: 


Developmental status 


CCSSE Item 


Q2M Q3IV 


Student goals 


CCSSE Item 


Q2M Q3IV 


Peer and family support 


CCSSE Item 


Q2M Q3IV 


Concurrent enrollment 


CCSSE Item (derived) 


Q2M Q3IV 


First-generation status 


CCSSE Item (derived) 


Q2M Q3IV 


Prior education 


CCSSE Item 


Q2M Q3IV 


Ethnicity 


CCSSE Item (derived) 


Q1IVQ2IV Q5IVQ7IV 


Immigrant status 


CCSSE Item 


Q4IV Q5IV Q7IV 


Weighting Variable 


Part- & full-time status weights 


CCSSE staff 


Applied to all analyses 



Analyses 

Research questions were examined using a variety of statistical techniques, including 
analysis of variance (ANOVA); correlations (Pearson product moment, point biserial, and phi 
coefficient) for estimating the relationships between two continuous variables, one continuous 
and one dichotomous variable, and two dichotomous variables, respectively; and regression 
analyses. For complete regression analysis results see Appendix Tables Cl - C6. 



Hierarchical regression models were used extensively in the analysis because they allow 
independent variables to be ordered according to their temporally or logically determined causal 
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priority or according to their research relevance, when some independent variables are the 
primary focus of the study (i.e., ethnicity and benchmark variables), but when other independent 
variables are also available (institution- and student-level variables). This procedure allows one 
to analyze the R Square, an estimate of the variance explained, for all independent variables or 
sets of variables in cumulative increments and to compare the proportion of dependent variable 
variance that is accounted for by the addition of each independent variable or set of variables to 
those higher in the hierarchy. Variables or sets of variables can be entered in a stepwise or 
forced-entry mode. In general, the hierarchical regression models with stepwise entry accounted 
for almost as much variance in transcript-derived student outcomes as did the forced-entry 
models. The stepwise models are thus preferred and discussed in the Results section because 
they are more parsimonious, using fewer variables to account for similar amount of variance in 
student outcomes. 



Results 

Overall differences in the levels of benchmarks, gain item clusters, and satisfaction 
between Hispanic and other students. There were significant differences between Hispanic and 
Non-Hispanic respondents on three of the five CCSSE benchmarks (Table 18). Hispanics 
reported slightly greater levels of Student Effort and Support for Learners and slightly less 
Student-Faculty Interaction. The differences in Student Effort and Student-Faculty Interaction, 
while statistically significant, were not noteworthy. Hispanic students reported significantly higher 
Academic, Personal Development, and Vocational Goals Gains. Both groups evaluated their 
experience at the community college very positively, and 96 percent of students reported that 
they would recommend their community college to a friend or family member. 
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Table 18 



Comparison of Hispanic vs. Non-Hispanic Students on Engagement, Gain, and Satisfaction 
Indices 



Hispanic Status 



CCSSE Construct 


Non-Hispanic 


Hispanic 






Total 




Mean 


N 


SD 


Mean 


N 


SD 


Mean 


N 


SD 


Active and Collaborative Learning 


.36 


9421 


.16 


.36 


3540 


.16 


.36 


12961 


.16 


Student Effort Scale* 


.45 


9421 


.16 


.47 


3540 


.15 


.46 


12962 


.16 


Academic Challenge Scale 


.55 


9417 


.17 


.55 


3540 


.17 


.55 


12957 


.17 


Student-Faculty Interaction Scale* 


.36 


9416 


.18 


.34 


3539 


.18 


.35 


12955 


.18 


Support for Learners Scale* 


.41 


9395 


.21 


.45 


3540 


.22 


.42 


12935 


.22 


Gains in Academics Factor* 


2.72 


9287 


.71 


2.86 


3509 


.69 


2.76 


12796 


.71 


Gains in Personal Development * 


2.26 


9265 


.83 


2.50 


3502 


.83 


2.33 


12767 


.84 


Gains in Vocational Goals Factor* 


2.46 


9293 


.87 


2.61 


3506 


.85 


2.50 


12799 


.86 


Recommend this college?* 


1.05 


9093 


.22 


1.03 


3527 


.17 


1.04 


12620 


.20 


Evaluate experience at this college* 


3.10 


9117 


.68 


3.17 


3535 


.66 


3.12 


12652 


.67 



* = p, .001 

Gain Indices: 1=Very Little, 2=Some, 3=Quite a Bit, 4=Very Much; Educational Experience: 
1=Poor, 2=Fair; 3=Good, and 4=Excellent; Recommend: 1=Yes, 2=No. 



Regression analyses were conducted with gain item clusters as outcomes. Ethnicity 
accounted for little additional variance in self-reported academic, personal development, and 
vocational goals after the influence of benchmarks were considered (Table 19). The Support for 
Learners and Academic Challenge benchmarks had by far the best predictive ability. 
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Table 19 



Engagement and Ethnicity Regressed on Gain Factors 











R Square 


Gain Index 


Regression Model 


R 


R Square 


Change 


Academics Factor 


5 Engagement Scales Without 


.631 


.398 






Ethnicity 

5 Engagement Scales With 


.633 


.401 


.003 




Ethnicity 

With only Academic Challenge 
& Support for Learners Scales 


.623 


.388 




Personal Development 


5 Engagement Scales Without 


.598 


.358 




Factor 


Ethnicity 

5 Engagement Scales With 


.605 


.366 


.008 




Ethnicity 

With only Academic Challenge 
& Support for Learners Scales 


.595 


.354 




Vocational Goals Factor 


5 Engagement Scales Without 


.596 


.355 






Ethnicity 

5 Engagement Scales With 


.597 


.356 


.002 




Ethnicity 

Engagement Scales With only 
Academic Challenge & Support 
for Learners Scales 


.590 


.348 





Support levels from the institution and faculty as predictors of differences between 
Hispanic and Non-Hispanic students. From early analyses, we learned that there were negligible 
differences between Hispanic and Non-Hispanic students on the five CCSSE benchmark 
variables and the two satisfaction variables. Even the three self-reported gain item clusters have 
relatively low correlations with Hispanic status: Academic Gains factor r= .09, Personal 
Development Gains factor r= .13, and Gains in Vocational Goals factor r= .08. Further analyses 
to attempt to account for such small group differences did not seem fruitful. 

Student-level and institution-level factors were used, in addition to CCSSE benchmarks, 
to help explain self-reported academic, personal development, and career-related Gains. Results 
indicate that there were similar patterns for academic, personal development, and career-related 
Gains (Table 20). The Academic Challenge and Support for Learners benchmarks had the best 
predictive ability, followed by student-level variables associated with educational goals 
(certificate, degree, or transfer) and quality of relationships with other students, instructors, and 
college personnel. Adding institution-level variables or ethnicity did not increase our ability to 
predict Gains. 
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Table 20 



Model Summary - Academic, Personal Development, and Career-Related Gains 





Gains in 




Gains in 




Academics 


Gains in Personal 


Vocational Goals 




Adjusted R 


Development 


Adjusted R 


Model 


Square 


Adjusted R Square 


Square 


1. CCSSE Benchmarks: 
Support for Learners 
Active and Collaborative 
Learning 


.411* 


.364* 


.368* 


Student Effort 
Student-Faculty Interaction 
Academic Challenge 








2. CCSSE Benchmarks and 
Institution Level Predictors: 


.414* 


.369* 


.370* 


Graduation Rate 
Total cohort 
Location 

IPEDS % Part-Time 
Enrollment 








3. CCSSE Benchmarks, 
Institution Level Predictors and 
Student Level Predictors: 


.455* 


.410* 


.420* 


Concurrent Enrollment 
First Generation Status 
Highest Academic Credential 
Career Change 
Developmental Math Course 
Transfer to 4 Year College 
Family Support for College 
Began at Current College 
ESL Course 
Complete Certificate 








Program 








Study Skills Course 
Other Students 
Self Improvement Courses 
Obtain an Associate Degree 
Developmental Reading 








Course 








Administrative Personnel 








Friends Support for College 
Instructors 








Developmental Writing 
Course 

4. CCSSE Benchmarks, 
Institution Level Predictors, 
Student Level Predictors and 
Ethnicity 


.456 


.409 


.421 



p< .001 
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International vs. US Born Differences on CCSSE Benchmarks, Gain Item Clusters, and 
Satisfaction. International students reported being significantly more engaged than US-born 
students on four CCSSE benchmarks (Table 21). The group differences were greatest on Student 
Effort and Support for Learners. International students reported significantly higher Academic, 
Personal Development, and Vocational Goals Gains than did their US-born peers. Both groups 
evaluated their experience at the community college very positively, and 96 percent of students 
reported that they would recommend their community college to a friend or family member. 



Table 21 

Comparison of Means on Engagement, Gain, and Satisfaction Indices: International vs. 
US Born 



Are you an international student or foreign 
national? 



CCSSE Construct 




Yes 






No 






Total 




Mean 


N 


SD 


Mean 


N 


SD 


Mean 


N 


SD 


Active and Collaborative Learning* 


.38 


1078 


.17 


.36 


11480 


.16 


.36 


12559 


.16 


Student Effort Scale* 


.52 


1079 


.16 


.45 


11480 


.16 


.46 


12559 


.16 


Academic Challenge Scale* 


.59 


1079 


.17 


.55 


11480 


.17 


.55 


12559 


.17 


Student-Faculty Interaction Scale 


.36 


1079 


.19 


.35 


11479 


.18 


.35 


12557 


.18 


Support for Learners Scale* 


.47 


1078 


.23 


.42 


11474 


.21 


.42 


12552 


.22 


Gains in Academics Factor* 


2.96 


1067 


.70 


2.73 


11374 


.70 


2.75 


12441 


.71 


Gains in Personal Development * 


2.66 


1064 


.85 


2.29 


11354 


.83 


2.32 


12418 


.84 


Gains in Vocational Goals Factor* 


2.65 


1068 


.86 


2.49 


11381 


.86 


2.50 


12449 


.87 


Recommend this college? 


1.05 


1071 


.23 


1.04 


11403 


.20 


1.04 


12473 


.20 


Evaluate your educational experience 


3.13 


1069 


.67 


3.12 


11441 


.67 


3.12 


12510 


.67 



* = p<.001 

Gain Indices: 1=Very Little, 2=Some, 3=Quite a Bit, 4=Very Much; Educational Experience: 
1=Poor, 2=Fair; 3=Good, and 4=Excellent; Recommend: 1=Yes, 2=No. 



International vs. US Born Differences on CCSSE Benchmarks, Gain Item Clusters, and 
Satisfaction. When the Hispanic status and immigrant status variables were combined to yield 
four groups (Table 22), Non-Hispanic immigrants reported significantly higher levels of 
engagement on four of the five benchmarks: Active and Collaborative Learning, Student Effort, 
Academic Challenge, and Student-Faculty Interaction. Overall, Non-Hispanic non-immigrants 
reported the least Academic, Personal Development, and Vocational Goals Gains. Hispanic and 
Non-Hispanic international students reported the most (and almost identical) Academic, Personal 
Development, and Vocational Goals Gains. Hispanic international students were more satisfied 
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with their community college experience than were the other three groups, although all groups 
reported very positive community college experiences. 



Table 22 

Comparison of Means on Engagement, Gain, and Satisfaction Indices: International 
Status within Ethnicity 









International within Ethnicity 












Hispanic, 

International 


Non-Hispanic 

International 


Hispanic, Not 
International 


Non- 

Hispanic, Not 
International 


Total 


CCSSE Variable 


Mean 


N 


Mean 


N 


Mean 


N 


Mean 


N 


Mean 


N 


Active and Collaborative 
Learning 


.36 


408 


.39 


670 


.36 


3101 


.36 


8380 


.36 


12559 


Student Effort 


.50 


408 


.53 


671 


.47 


3101 


.44 


8380 


.46 


12559 


Academic Challenge 


.58 


408 


.60 


671 


.55 


3101 


.55 


8380 


.55 


12559 


Student-Faculty Interaction 


.35 


408 


.37 


671 


.34 


3099 


.36 


8380 


.35 


12557 


Support for Learners 


.48 


408 


.47 


670 


.45 


3100 


.41 


8374 


.42 


12552 


Gains in Academics 


2.97 


405 


2.96 


662 


2.84 


3072 


2.69 


8302 


2.75 


12441 


Gains in Personal 
Development 


2.70 


402 


2.63 


662 


2.47 


3068 


2.22 


8286 


2.32 


12418 


Gains in Vocational Goals 


2.67 


405 


2.64 


663 


2.61 


3069 


2.44 


8312 


2.50 


12449 


Recommend this college? 


1.04 


405 


1.06 


666 


1.03 


3091 


1.05 


8312 


1.04 


12473 


Evaluate your educational 
experience 


3.21 


407 


3.09 


662 


3.16 


3096 


3.11 


8345 


3.12 


12510 



All variables p < .001 for F-Tests 

Gain Indices: 1=Very Little, 2=Some, 3=Quite a Bit, 4=Very Much; Educational Experience: 
1=Poor, 2=Fair; 3=Good, and 4=Excellent; Recommend: 1=Yes, 2=No. 



Immigrant status accounted for little additional variability in self-reported academic, 
personal development, and vocational goals after the influence of student engagement factors 
was considered (Table 23). The Support for Learners and Academic Challenge benchmarks had 
by far the best predictive ability. 
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Table 23 



Engagement and Immigrant Status (IS) Regressed on Gain Item Clusters 











R Square 


Gain Item Cluster 


Regression Model 


R 


R Square 


Change 


Academics Factor 


5 Engagement Scales Without 
IS 

5 Engagement Scales With IS 


.631 


.398 






.631 


.399 


.000 




With only Academic Challenge 
& Support for Learners Scales 


.622 


.387 




Personal Development 


5 Engagement Scales Without 


.598 


.358 




Factor 


IS 

5 Engagement Scales With IS 


.603 


.364 


.006 




With only Academic Challenge 
& Support for Learners Scales 


.595 


.354 




Vocational Goals Factor 


5 Engagement Scales Without 
IS 

5 Engagement Scales With IS 


.597 


.357 






.598 


.357 


.000 




Engagement Scales With only 
Academic Challenge & Support 
for Learners Scales 


.590 


.348 





International Status and Ethnicity as Predictors of Transcript-Derived Student Outcomes. 
Differences between means for six transcript-derived student outcomes broken down by 
international status within ethnicity were analyzed (Table 24). For cumulative GPA, there was a 
main effect for international status where international students had higher cumulative GPAs than 
non-international students. For one-year persistence (first to third term), there was an interaction 
effect: Non-Hispanic, international students had higher persistence rates than Hispanic, 
international students while there were no significant differences between Hispanic and Non- 
Hispanic non-international students. For the outcome measure total credit hours taken, there 
were main effects for both ethnicity (Non-Hispanics had more total credit hours) and for 
international status (immigrants had more total credit hours). For the outcome measure average 
credit hours, there were main effects (international higher and Non-Hispanic higher) and an 
interaction effect, with the differences between Hispanic and Non-Hispanic students much greater 
for international students than for non-international students. 
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Table 24 



Comparison of Means on Transcript-Derived Student Outcomes: International Status within 
Ethnicity 



International within 
Ethnicity 




Cum 

GPA 


1st to 2nd 
Term 

Persistence 


1 st to 3rd 
Term 

Persistence 


Total 

Credit 

Hours 

Taken 


Number 

of 

Terms 

Enrolled 


Average 

Credit 

Hours 


Hispanic, International 


Mean 


2.90 


.90 


.78 


45.07 


4.93 


9.18 




N 


90 


89 


88 


90 


90 


90 




S.D. 


.82 


.30 


.42 


26.99 


2.67 


2.78 


Non-Hispanic 


Mean 


2.90 


.95 


.88 


55.84 


5.09 


10.85 


International 


N 


199 


195 


195 


199 


199 


199 




S.D. 


.72 


.23 


.32 


30.34 


2.61 


3.44 


Hispanic, Not 


Mean 


2.73 


.93 


.79 


41.61 


4.64 


9.09 


International 


N 


989 


950 


945 


994 


994 


994 


Non-Hispanic, Not 


S.D. 


.74 


.25 


.41 


24.20 


2.58 


3.24 


International 


Mean 


2.86 


.93 


.81 


47.19 


5.00 


9.67 




N 


1919 


1893 


1898 


1928 


1928 


1928 




S.D. 


.77 


.25 


.40 


27.12 


2.79 


3.27 


Total 


Mean 


2.82 


.93 


.80 


45.94 


4.89 


9.55 




N 


3197 


3127 


3125 


3211 


3211 


3211 




S.D. 


.76 


.25 


.40 


26.69 


2.72 


3.28 



Persistence and GPA models assessed the extent to which CCSSE benchmarks and 
item clusters predict these outcomes in addition to examining the impact of Hispanic and Non- 
Hispanic students and immigrant and US born students (Table 25). 



Table 25 

F Values for 2 x 2 ANOVA of Student Outcomes on Ethnicity by International Status 







1 st to 2 nd 


1 st to 3 rd 


Total 




Average 






Credit 


Number 


Credit 




Cumulative 


Term 


Term 


Hours 


of Terms 


Hours 


Source 


GPA 


Persistence 


Persistence 


Taken 


Enrolled 




Ethnicity 


2.283 


2.107 


7.535** 


25.339*** 


2.954 


26.789*** 


International 

Status 


5.002* 


.448 


1.290 


11.481*** 


1.012 


8.758** 


Ethnicity * 
International 


1.286 


2.111 


3.958* 


2.710 


.126 


5.825* 


Status 














*p < .05, **p < 


.01, ***p< .001 
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Based on results indicating that ethnicity, international status, and the interaction 
between the two variables proved significant, the ethnicity by international status interaction term, 
in addition to CCSSE benchmarks and satisfaction items, was used to help predict the six 
transcript-derived student outcomes. 

For cumulative GPA, the Support for Learners and Academic Challenge benchmarks, 
willingness to recommend college to friends or family, and the Hispanic status by international 
status interaction variable contributed significantly to the prediction of cumulative GPA. Model 
results are presented in Table 26. 



Table 26 

Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Perceived Gain Item 
Clusters, Satisfaction Variables, and Hispanic/International Status Regressed on Cumulative 
GPA 







R Square 




Std. Error of 


Model 


R 


(R Square Change) 


Adjusted R Square 


the Estimate 


1 . Student Effort 


.123 


.015 


.015 


.757 


2. Model 1 predictor 
and Support for 
Learners 


.138 


.019 

(.004) 


.018 


.756 


3. Model 2 predictors 
and Academic 


.157 


.025 

(.006) 


.024 


.754 


Challenge 

4. Model 3 predictors 
and Education 


.257 


.065 


.738 


.066 

(.041) 


Experience Evaluation 
5. Model 4 predictors 
and Recommend 


.263 


.068 


.737 


.069 

(.003) 


College Friend/Family 
6. Model 5 predictors 
and Ethnicity X 
International Status 


.273 


.073 


.735 


.074 

(.005) 



The Active and Collaborative Learning and Student-Faculty Interaction benchmarks were 



the strongest predictors of first to second term persistence. Overall, 93 percent of the students in 
the sample persisted from the first to second term. Thus, there was very little variance to predict 
in this outcome measure. Model results are presented in Table 27. 
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Table 27 



Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Gain Items, Satisfaction 
Variables, and Hispanic/International Status Regressed on First to Second Term Persistence 







R Square 




Std. Error of 


Model 


R 


(R Square Change) 


Adjusted R Square 


the Estimate 


1 . Active and 
Collaborative Learning 


.109 


.012 


.012 


.248 


2. Model 1 predictor 
and Student-Faculty 
Interaction 


.115 


.013 

(.001) 


.013 


.248 


3. Model 2 predictors 
and Gains in 


.127 


.016 

(.003) 


.015 


.247 


Academics 
4. Model 3 predictors 
and Education 
Experience Evaluation 


.136 


.017 


.247 


.019 

(.003) 



For first to third term (i.e., one year) persistence, Active and Collaborative Learning, 
Support for Learners, Gains in Academics, and Hispanic Status made significant contributions to 
the prediction of first to third term persistence. Model results are presented in Table 28. 



Table 28 

Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Gain Items, Satisfaction 
Variables and Hispanic/International Status Regressed on First to Third Term Persistence 



Model 


R 


R Square 

(R Square Change) 


Adjusted R Square 


Std. Error of 
the Estimate 


1 . Active and 
Collaborative Learning 


.112 


.013 


.012 


.394 


2. Model 1 predictor and 
Support for Learners 
Scale 


.119 


.014 

(.001) 


.014 


.394 


3. Model 2 predictors and 
Gains in Academics 


.140 


.020 

(.006) 


.019 


.393 


4. Model 3 predictors and 


.145 


.021 


.020 


.393 


Ethnicity 




(.001) 







For total credit hours taken, student satisfaction contributed little to prediction after 
benchmarks and gain item clusters were taken into account. Seven predictors: Active and 
Collaborative Learning, Student-Faculty Interaction, Support for Learners, Gains in Academics, 
Race by International Status Interaction, International Status, and Hispanic Status were 

significant predictors of total credit hours taken. Model results are presented in Table 29. 
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Table 29 



Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Gain Items, Satisfaction 
Variables, and Hispanic/International Status Regressed on Total Credit Hours Taken 







R Square 




Std. Error of 


Model 


R 


(R Square Change) 


Adjusted R Square 


the Estimate 


1 . Active and 
Collaborative Learning 


.177 


.031 


.031 


26.30 


2. Model 1 predictor and 

Student-Faculty 

Interaction 


.198 


.039 

(.008) 


.039 


26.20 


3. Model 2 predictors and 


.205 


.042 


.041 


26.16 


Support for Learners 




(.003) 






4. Model 3 predictors and 


.237 


.056 


.055 


25.98 


Gains in Academics 




(.014) 






5. Model 4 predictors and 
Ethnicity X International 
Status 


.262 


.069 

(.013) 


.067 


25.81 


6. Model 5 predictors and 


.267 


.071 


.069 


25.78 


International Status 




(.002) 






7. Model 6 predictors and 


.270 


.073 


.071 


25.76 


Ethnicity 




(.002) 







The number of terms enrolled was predicted by Active and Collaborative Learning, 
Student-Faculty Interaction, Gains in Academics, and Ethnicity by International Status Interaction. 
Model results are presented in Table 30. 

Table 30 

Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Gain Items, Satisfaction 



Variables, and Hispanic/International Status Regressed on Numbers of Terms Enrolled 



Model 


R 


R Square 

(R Square Change) 


Adjusted R Square 


Std. Error of 
the Estimate 


1 . Active and 
Collaborative Learning 


.119 


.014 


.014 


2.698 


2. Model 1 predictor and 

Student-Faculty 

Interaction 


.131 


.017 

(.003) 


.016 


2.694 


3. Model 2 predictors and 
Gains in Academics 


.167 


.028 

(.011) 


.027 


2.680 


4. Model 3 predictors and 
Ethnicity X International 
Status 


.179 


.032 

(.004) 


.031 


2.675 



Average Credit Hours Taken was predicted by Student-Faculty Interaction, Support for 
Learners, Active and Collaborative Learning, Academic Challenge, Gains in Vocational Goals, 
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Gains in Academics, both satisfaction items, Ethnicity, International Status, and Ethnicity by 
International Status Interaction. Model results are presented in Table 31 . 



Table 31 

Stepwise Entry within Blocks Model Summary for CCSSE Benchmarks, Gain Items, Satisfaction 
Variables, and Hispanic/International Status Regressed on Average Credit Hours Taken 







R Square 




Std. Error of 


Model 


R 


(R Square Change) 


Adjusted R Square 


the Estimate 


1. Student-Faculty 
Interaction 


.159 


.025 


.025 


3.24 


2. Model 1 predictor and 


.178 


.032 


.031 


3.23 


Support for Learners 




(.007) 






3. Model 2 predictors and 
Active and Collaborative 


.188 


.035 


.034 


3.22 


Learning 




(.003) 






4. Model 3 predictors and 


.191 


.037 


.035 


3.22 


Academic Challenge 




(.002) 






5. Model 4 predictors and 


.201 


.040 


.039 


3.22 


Gains in Vocational Goals 




(.003) 






6. Model 5 predictors and 


.211 


.044 


.043 


3.21 


Gains in Academics 
7. Model 6 predictors and 
Recommend College 


.228 


(.004) 

.052 

(.008) 


.050 


3.20 


Friend/Family 
8. Model 7 predictors and 
Education Experience 
Evaluation 


.234 


.052 


3.19 


.055 

(.003) 


9. Model 8 predictors and 


.256 


.065 


.063 


3.18 


Ethnicity 




(.01) 






10. Model 9 predictors 


.264 


.070 


.067 


3.17 


and International Status 




(.005) 






11. Model 10 predictors 
and Ethnicity X 
International Status 


.267 


.072 

(.002) 


.068 


3.17 



CCSSE-Reported Outcomes as Proxies for Transcript-Derived Outcomes. A number of 
CCSSE constructs were good proxies for transcript-derived student outcome variables (Table 
32). The correlation between CCSSE self-reported Grade Average range and transcript-derived 
cumulative GPA was an impressive .55. And, the CCSSE variable, Total Credit Hours Earned, 
correlated highly with the transcript-derived variables Total Credit Hours Taken (r= .57) and 
Number of Terms Enrolled ( r= .55). 
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Table 32 



Transcript-Derived Outcomes Correlations with CCSSE-Reported Outcomes 



Outcome Measure 




Your overall 
college 
grade 
average? 


TOTAL credit 
hours earned 
at this 
college? 


Gains in 
Academics 
Factor 


Gains in 
Personal 
Development 
Factor 


Gains in 
Vocational 
Goals 
Factor 


Cumulative GPA 


Pearson r 


.548(“) 


.1 02(“) 


.057(“) 


-.002 


.020 




N 


3203 


3215 


3221 


3218 


3223 


1st to 2nd Term 


Pearson r 


.1 01 (“) 


.281 (“) 


.092(“) 


.083(“) 


,068(“) 


Persistence 


N 


3135 


3147 


3154 


3151 


3155 


1st to 3rd Term 


Pearson r 


.1 15(“) 


.392(“) 


.1 14(“) 


.097(“) 


.074(“) 


Persistence 


N 


3133 


3145 


3152 


3149 


3153 


Total Credit Hours 


Pearson r 


,100(“) 


.566(“) 


.1 91 (“) 


■1 59(“) 


■11 6(“) 


Taken 


N 


3217 


3230 


3236 


3232 


3237 


Number of Terms 


Pearson r 


.1 03(“) 


.548(“) 


.140(“) 


.1 24(“) 


,092(“) 


Enrolled 


N 


3217 


3230 


3236 


3232 


3237 


Average Credit Hours 


Pearson r 


.054(“) 


■146(“) 


■1 33(“) 


.095(“) 


,053(“) 




N 


3217 


3230 


3236 


3232 


3237 



‘‘Correlation is significant at the 0.01 level (2-tailed). 



Discussion 

The CCSSE benchmarks were good predictors of both CCSSE self-reported outcomes 
and transcript-derived student outcomes. Overall, two benchmarks, Academic Challenge and 
Support for Learners, were the best and most consistent predictors of student outcomes. After 
considering the effects of student engagement, when self-reported academic Gains and 
satisfaction items were added as either predictors or moderator variables, self-reported Gains 
tended to add little to our ability to predict outcomes, whereas satisfaction makes an independent 
contribution. This is because Academic, Personal Development, and Vocational Goals Gain items 
were more highly correlated (i.e., share more common variance) with benchmarks than were the 
two satisfaction variables; thus, the satisfaction items make an independent contribution to the 
prediction of outcomes while gain items did not. 
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Immigrant status should definitely be accounted for in any future CCSSE research. 
Immigrant students reported much more Student Effort, Academic Challenge, Support for 
Learners, Academic Development, Personal Development, and Vocational Goals Gains than did 
non-immigrants. And, immigrant students were not a homogeneous group. There were many 
differences between Hispanic and Non-Hispanic immigrants. Further, in a number of the 
regression analyses, the Hispanic status by immigrant status interaction term was a significant 
(but not noteworthy) predictor of transcript-derived student outcomes. We suspect if other 
demographic variables were examined (e.g., gender, age, marital status) that other interactive 
effects would be found. 

In the current study, student-level variables, such as those associated with educational 
goals, were better predictors of student outcomes than were institution-level factors. Only four 
IPEDS-derived institutional variables were included in this study. A future CCSSE study might 
incorporate a full range of IPEDS information. And, after four years of administration, the CCSSE 
database is large enough to employ institution as the unit of analysis rather than individual 
students. This method would allow for a more robust test of the influence of institution-level 
variables on student outcomes. 

Upon reflection, the decision to select only HSS institutions may have muted our ability to 
study certain phenomena. Since the study institutions have a “critical Hispanic mass” (IPEDS 
average for 16 institutions is 28.2 percent Hispanic), it may be that student support services and 
curriculum are more geared and oriented toward Hispanic students in this sample than at 
community colleges with smaller proportions of Hispanic students. The research questions 
addressed in this study that involve only CCSSE variables could be better addressed employing 
the whole CCSSE database, resulting in greatly increased sample sizes and increased variation 
on some variables of interest. 

The results clearly demonstrate that in assessing the “validity” of the CCSSE, the choice 
of student outcomes variables is very important. We were able to account for larger proportions of 
variance in cumulative GPA, Total Credit Hours Taken, and Average Credit Hours Earned than in 
First to Second Term Persistence, First to Third Term Persistence, and Number of Terms 
Enrolled. Further, depending on the student outcome of interest, some CCSSE self-reported 
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outcomes seemed to be good proxies for transcript-derived outcomes, specifically cumulative 
GPA and Total Credit Hours Earned. 

Overall, many of the CCSSE variables and corresponding derived scales and factors, 
demonstrated solid relationships with both self-reported and transcript-derived student outcomes. 
And, although validity is often “in the eyes of the beholder,” the evidence from this study, 
especially given its methodological limitations, suggests that the CCSSE has good validity. 
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SUMMARY ACROSS VALIDATION STUDIES 



The results of the three studies broadly support the impact of engagement on students’ 
academic outcomes. A wide variety of academic outcomes, including Cumulative GPA, Number 
of Terms Enrolled, Credit Completion Ratio, Total Credit Hours Completed, First to Second Term 
Persistence, and First to Second Year Persistence were examined and were consistently related 
to student engagement across all studies. While support for relationships between engagement 
and academic outcomes was broad, there were some measures that produced more consistent 
results than others. Cumulative measures of enrollment and credit hours accumulated exhibited 
the strongest relationships with engagement. Course performance measures, including GPA and 
Credit Completion Ratio, and measures of persistence were also consistently related to 
engagement measures in predicable ways. In addition, there were several study-specific 
measures, such as attainment of college pathway status and transfer readiness, that provide 
strong support for the proposition that student engagement matters in student success. 

Bivariate Relationships between CCSSE Predictors and Performance Measures 

There was considerable overlap in the outcome measures across the three studies. To 
evaluate consistencies across studies, we began by examining the bivariate correlations between 
CCSSE benchmarks, item clusters, and gain items to identify consistent patterns in relationships 
across studies. Results of bivariate correlations are presented in Table 33. For purposes of 
discussing correlation results from Table 33 the term “strong” refers to CCSSE constructs that 
were significant predictors of an outcome measure across all three studies, “good” refers to 
CCSSE constructs that were significant predictors of an outcome measure across two studies, 
and “adequate” refers to CCSSE constructs that were significant predictors of an outcome 
measure in one study. 

Academic Measures 

Cumulative GPA. Across all three studies, the relationships between Cumulative GPA 
and CCSSE constructs were examined. Student-Faculty Interaction was a good predictor; and 
Active and Collaborative Learning, Student Effort, and Academic Challenge were strong 
predictors of Cumulative GPA. Four item clusters (Class Assignments, Collaborative Learning, 
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Information Technology, and Student Services) were adequate predictors, while four other item 
clusters (Faculty Interactions, Exposure to Diversity, Mental Activities, and Academic Preparation) 
were strong predictors of Cumulative GPA. The perceived Gain in Academics item was a good 
predictor of Cumulative GPA. 

Credit Completion Ratio. Correlations between Credit Completion Ratio and CCSSE 
constructs were analyzed for the Achieving the Dream and Florida studies. Active and 
Collaborative Learning and Academic Challenge benchmarks were both good predictors, while 
Student Effort and Student-Faculty Interaction benchmarks were adequate predictors of Credit 
Completion Ratio. Class Assignments, Collaborative Learning, Mental Activities, and Academic 
Preparation item clusters were good predictors of Credit Completion Ratio; Faculty Interactions 
and Information Technology item clusters were adequate predictors. The Academic Gain item 
cluster was a good predictor of Credit Completion Ratio. 

Persistence Measures 

First to Second Term Persistence. Across all three studies, the relationships between 
First to Second Term Persistence and CCSSE constructs were examined. All five benchmarks, 
Active and Collaborative Learning (strong predictor), Student Effort, Student-Faculty Interaction, 
and Support for Learners (good predictors), and Academic Challenge (adequate predictor), had a 
statistically significant relationship with First to Second Term Persistence in at least one study. 
Class Assignments, Exposure to Diversity, School Opinions, and Academic Preparation item 
clusters adequately predicted First to Second Term Persistence. Faculty Interactions, 
Collaborative Learning, and Information Technology item clusters were good predictors, and the 
Student Services item cluster was a strong predictor of First to Second Term Persistence. The 
Vocational Goal Gain item cluster was a good predictor, and the Academic Gain and Personal 
Development Gain item clusters were strong predictors of First to Second Term Persistence. 

First to Second Year Persistence. The relationships between First to Second Year 
Persistence and CCSSE constructs were examined across studies. The Active and Collaborative 
Learning benchmark was a strong predictor; Student Effort and Support for Learners benchmarks 
were good predictors, and Academic Challenge and Student-Faculty Interaction benchmarks 
were both adequate predictors of First to Second Year Persistence. Each of the engagement 
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item clusters was at least an adequate predictor of First to Second Year Persistence. Perceived 
gain item clusters were adequate (Gains in Personal Development), good (Gains in Vocational 
Goals), and strong (Gains in Academics) predictors of First to Second Year Persistence. 

Degree/Certificate Completion. Correlations between Degree/Certificate Completion and 
CCSSE constructs were analyzed for the Achieving the Dream and Florida studies. The Support 
for Learners benchmark was an adequate predictor, and Active and Collaborative Learning, 
Academic Challenge, and Student-Faculty Interaction benchmarks were good predictors of 
Degree/Certificate Completion. The Class Assignments item cluster was an adequate predictor, 
and Faculty Interactions, Collaborative Learning, Information Technology, Mental Activities, and 
Academic Preparation item clusters were good predictors of Degree/Certificate Completion. The 
perceived Gains in Academics item cluster was an adequate predictor of Degree/Certificate 
Completion. 

Longevity Measures 

Number of Terms Enrolled. All five benchmarks were strong predictors of Number of 
Terms Enrolled. The Academic Preparation item cluster was an adequate predictor, the 
Exposure to Diversity item cluster was a good predictor, and the remaining item clusters were 
strong predictors of Number of Terms Enrolled. All three perceived gain item clusters were also 
strong predictors of Number of Terms Enrolled. 

Total Credit Hours Completed. All five benchmarks were strong predictors of Total Credit 
Hours Completed. Faculty Interactions and Student Services item clusters were good predictors, 
and the remaining item clusters were strong predictors of Total Credit Hours Completed. 
Perceived Personal Development and Vocational Goal Gain item clusters were both good 
predictors, and the Academic Gain item was a strong predictor of Total Credit Hours completed. 
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Table 33 



Bivariate Correlations between Outcome Measures and CCSSE Constructs 





Number of Terms Enrolled 


Total Credit Hours Completed 


Credit Completion 


CCSSE 














Ratio 




Predictor 


Achieving 


HSS 


Florida 


Achieving 


HSS 


Florida 


Achieving 


Florida 




the Dream 






the Dream 






the Dream 




Active and 
Collaborative 
Learning 
Student 


.128*** 


-| 2 -j *** 


.118*** 


.225*** 


.178*** 


.159*** 


1 22 *** 


0.070** 


Effort 

Academic 


.093*** 


.065*** 


.117*** 


.155*** 


.114*** 


.113*** 


.106*** 


0.006 


Challenge 

Student- 


.099*** 


.066*** 


.123*** 


.197*** 


.131*** 


.137*** 


-| 2 -j *** 


0.070** 


Faculty 

Interaction 


. 102 *** 


.116*** 


.151*** 


.197*** 


.175*** 


.105*** 


.083*** 


-0.004 


Support for 
Learners 


.113*** 


.060*** 


.084*** 


.094*** 


.124*** 


.035* 


-.045 


0.031 


Faculty 

Interactions 


.087*** 


.104*** 


. 101 *** 


.188*** 


.136*** 


.017 


.105*** 


0.009 


Class 

Assignments 


.108*** 


.092*** 


.126*** 


.186*** 


.153*** 


.146*** 


■\ -) 4 *** 


0.056* 


Exposure to 

Diversity 

Collaborative 


.010 


.079*** 


.048** 


.077** 


.130*** 


. 100 *** 


.031 


0.008 


Learning 

Information 


.125*** 


.092*** 


.077*** 


.197*** 


.166*** 


. 102 *** 


.063* 


0.051* 


Technology 

Mental 


.064* 


.038* 


.132*** 


.158*** 


■j 22 *** 


.134*** 


.086*** 


0.012 


Activities 


.096*** 


.066*** 


.084*** 


.169*** 


.104*** 


.073*** 


.106*** 


0.052* 


School 

Opinions 

Student 


.091*** 


.049** 


.061*** 


.089*** 


.113*** 


.053** 


-.040 


0.034 


Services 

Academic 


■\ 42 *** 


.079*** 


.093*** 


.125*** 


.134*** 


.011 


-.001 


-0.017 


Preparation 


.040 


.016 


.153*** 


.194*** 


■j 2 "j *** 


.248*** 


.128*** 


0.090*** 


Gains in 
Academics 


.155*** 


.140*** 


.117*** 


.218*** 


.191*** 


. 120 *** 


.078** 


0.082*** 


Gains in 

Personal 

Development 


. 121 *** 


.124*** 


.076*** 


.117*** 


.159*** 


.011 


-.030 


-0.006 


Gains in 

Vocational 

Goals 


.088*** 


.092*** 


.109*** 


.126*** 


.116*** 


.033 


-.019 


0.040 
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Table 33 (continued) 



CCSSE 


First to Second Term Persistence 


First to Second Year Persistence 


Predictor 


Achieving 
the Dream 


HSS 


Florida 


Achieving 
the Dream 


HSS 


Florida 


Active and 

Collaborative 

Learning 


-.052* 


. 110 *** 


.063** 


.059* 


■j -j 2 *** 


0.085*** 


Student Effort 


.078** 


.048** 


.044 


.058* 


.038* 


0.029 


Academic 

Challenge 


-.005 


.038* 


.025 


.038 


.054** 


0.025 


Student- 

Faculty 

Interaction 


-.066** 


.093*** 


.004 


-.019 


.094*** 


0.041 


Support for 
Learners 


.038 


.052** 


.052* 


.047 


.070*** 


0.053* 


Faculty 

Interactions 


-.088*** 


.081*** 


.015 


-.042 


.081*** 


0.040 


Class 

Assignments 


.023 


.103*** 


.044 


.064* 


.084*** 


0.077*** 


Exposure to 
Diversity 


-.027 


.067*** 


.008 


.031 


.045** 


-0.006 


Collaborative 

Learning 


-.040 


.090*** 


.068** 


.066* 


.106*** 


0.085*** 


Information 

Technology 


-.022 


.047** 


-.061** 


.063* 


.049** 


0.031 


Mental 

Activities 


-.009 


.019 


.006 


.042 


.036* 


0.005 


School 

Opinions 


.027 


.041* 


.028 


.034 


.058*** 


0.039 


Student 

Services 


.103*** 


.055** 


.082*** 


.079** 


.069*** 


0.043 


Academic 

Preparation 


.013 


.044* 


.038 


.021 


.050** 


0.044* 


Gains in 
Academics 


■j 2 "j *** 


.092*** 


.051* 


-| ^ -| *** 


■\ -| 4 *** 


0.057* 


Gains in 

Personal 

Development 


.055* 


.083*** 


.048* 


.040 


.097*** 


0.033 


Gains in 

Vocational 

Goals 


-.005 


.068*** 


.060** 


.062* 


.074*** 


0.024 
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Table 33 (continued) 



CCSSE 

Predictor 




Cumulative GPA 




Degree/Certificate 

Completion 




Achieving 
the Dream 


HSS 


Florida 


Achieving 
the Dream 


Florida 


Active and 

Collaborative 

Learning 


■\ 4-1 *** 


0.082*** 


0.115*** 


.101*** 


.107*** 


Student Effort 


.059* 


0.119*** 


0.044* 


-.008 


.013 


Academic 

Challenge 


.100*** 


0.103*** 


0.077*** 


.069** 


.073** 


Student- 

Faculty 

Interaction 


.090** 


0.077*** 


0.031 


.110*** 


.071** 


Support for 
Learners 


-.028 


-0.02 


0.017 


-.021 


.054* 


Faculty 

Interactions 


.121*** 


0.117*** 


0.087*** 


.108*** 


.074** 


Class 

Assignments 


.055 


0.05** 


0.024 


.030 


.068** 


Exposure to 
Diversity 


.072* 


0.045** 


0.067** 


.030 


.032 


Collaborative 

Learning 


.098*** 


0.011 


0.040 


.088*** 


.104*** 


Information 

Technology 


.058 


0.046** 


0.020 


.064** 


.057* 


Mental 

Activities 


.084** 


0.095*** 


0.094*** 


.061* 


.052* 


School 

Opinions 


-.012 


-0.002 


0.040 


-.019 


.040 


Student 

Services 


-.047 


-0.02 


-0.088*** 


-.022 


.009 


Academic 

Preparation 


-j 2^*** 


0.124*** 


0.075*** 


.080*** 


.095*** 


Gains in 
Academics 


.055 


.057*** 


0.066** 


.009 


.088*** 


Gains in 

Personal 

Development 


-.016 


-0.002 


-0.043 


.006 


.013 


Gains in 

Vocational 

Goals 


-.012 


0.02 


0.009 


.072** 


.026 



*p< .05, **p< .01, ***p< .001 
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Patterns across Studies 



Academic Measures 

The two outcomes that are most prototypically academic were Cumulative GPA and 
Credit Completion Ratio. GPA was analyzed by each of the three studies and considered in a 
number of different ways. Both the Florida and Achieving the Dream studies examined Credit 
Completion Ratio as an outcome measure. 

Full Cohort and Cross-sectional performance file analyses results for the Florida 
validation study show that CCSSE constructs are significant bivariate and net predictors of 
college-level GPA. Each of the “academic” CCSSE item clusters (including all benchmarks) and 
the Academic Gain item were significantly associated with Cumulative GPA net effects in the full 
cohort model; these results were replicated in the Cross-Sectional file with the exception of 
Support for Learners. Support for Learners consistently failed to exhibit a significant bivariate 
relationship with GPA in the Florida sample, consistent with earlier studies (Marti, in press). For 
Short cohort analyses, significant net effects on GPA within the first three terms of enrollment 
emerged only for Active and Collaborative Learning, Student Effort, and Class Assignments, 
while the validity of self-reported Academic Gains was again modestly confirmed; in contrast, only 
Student Effort emerged as significant in analyses of first year GPA in Short cohorts. This 
suggests that the net effects of engagement on academic outcomes is more marked in later 
terms of enrollment — after a student has achieved “college path” status — than in the first three 
terms of enrollment. 4 Although the results were stronger for first three terms than first year, this 
affirms the importance of engagement in students’ early experience. 

Achieving the Dream results exhibited a high degree of similarity with GPA analyses in 
the Florida study. CCSSE benchmarks positively predict Cumulative GPA after two years and 
Cumulative GPA at the end of the term in which CCSSE was administered. Overall, Active and 
Collaborative Learning, Academic Challenge and Student-Faculty Interaction had positive net 
effects when predicting Cumulative GPA, and all benchmarks other than Support for Learners 

4 Short cohorts also had significantly lower GPA than Long cohorts (2.84 vs. 3.01) reflecting both 
the superior academic performance for “survivors” and the typical phenomenon at most institution 
of increasing grades in later terms of enrollment for successful students. 
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exhibited significant bivariate relationships with GPA. Several CCSSE item clusters were also 
positive predictors of Cumulative GPA. 

The HSS study also shared similarity with GPA analyses in the Florida and Achieving the 
Dream studies in the bivariate relationships. For Cumulative GPA examined in the FISS study, 
three benchmarks (Student Effort, Support for Learners, Academic Challenge), and two item 
clusters contributed significantly to the prediction of Cumulative GPA. Other variables in these 
models included item clusters, gain items, and Hispanic and international status. Thus, use of 
this multivariate model decreased the strength of Active and Collaborative Learning and Student- 
Faculty Interaction while increasing the strength of the relationship between Support for Learners 
and GPA. 

Full cohort analyses results for Florida’s validation study show that CCSSE constructs 
(including the CCSSE Academic Gain item) are significant bivariate and net predictors of Credit 
Completion Ratio, but are somewhat less well associated after controls are introduced. For 
Credit Completion analyses using the Short cohort file, Class Assignments, Support for Learners, 
and School Opinions emerged as significant net predictors within the first three terms of 
enrollment. With regard to the Cross-sectional performance file analyses results, all of the 
CCSSE “academic" clusters and the Collaborative Learning and Student Services cluster items 
are related to three-term Credit Completion Ratios. Similar to the GPA analyses, the Support for 
Learners benchmark interacted with initial academic ability, meaning that students with lower 
initial levels of academic ability exhibited a positive relationship between the Student Effort 
benchmark and Credit Completion Ratios while students with higher initial levels of academic 
ability exhibited a negative relationship between Support for Learners and Credit Completion 
Ratio. In analyses examining the proportion of courses completed with a grade of C or better, full 
cohort and cross-sectional performance file analyses for the Florida validation study showed that 
CCSSE constructs (including the CCSSE Academic Gain item) are significant bivariate and net 
predictors of completion of courses with a grade of “C” or better, but are somewhat less well 
associated after controls are introduced. Academic Challenge, Academic Preparation and self- 
reported Gain in Academics showed significant net effects in predicting the proportion of courses 
completed with a grade of “C” or better in the first three terms of enrollment. The Achieving the 
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Dream validation study showed that four of the five CCSSE benchmarks - Active and 
Collaborative Learning, Student Effort, Academic Challenge, and Student-Faculty Interaction - 
had positive net effects when predicting cumulative Credit Completion Ratios; several item 
clusters and students’ perceived Academic Gains were also positive predictors of Credit 
Completion Ratios after two years. 

Bivariate correlations demonstrated strong consistency across GPA analyses for the 
three studies. Four benchmarks, all but Support for Learners, were significantly correlated with 
GPA, though there were mixed results across the Florida analytic files for Student-Faculty 
Interaction. There were four item clusters that exhibited significant correlations with GPA across 
all three studies: Faculty Interactions, Exposure to Diversity, Mental Activities, and Academic 
Preparation. These patterns held up with only minor exceptions in net effects for these factors in 
regression models. Furthermore, the Academic Gains item cluster exhibited significant effects in 
several models and several bivariate relationships, indicating that the Academic Gains item 
cluster is related to GPA. It is notable that while the Support for Learners benchmark consistently 
fails to exhibit significant bivariate relationships with GPA, it does emerge as significant in 
interactions with initial academic ability in the Florida study and in a multivariate regression model 
in the HSS study, suggesting a suppressor effect. This pattern suggests that Support for 
Learners may be more important for some students than others. 

Credit Completion Ratio wasn’t examined as thoroughly as GPA, but results were 
consistent with GPA, though not as strong. Each of the benchmarks, with the exception of 
Support for Learners, exhibited a bivariate relationship in at least one of the two studies 
examining Credit Completion Ratio and conditional effects for the Support for Learners 
benchmark. The strong consistency in the results across studies demonstrates that the 
academically related item clusters were consistent predictors of GPA and Credit Completion 
Ratio, and Student Effort may be conditionally related to these constructs as well. 
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Early Academic Measures 

There were several measures that pertain specifically to early academic experiences in 
college. These include success in developmental education and gatekeeper courses as 
measured by either course completion or grades in those courses. The Florida and Achieving the 
Dream studies both had data on developmental and gatekeeper courses. In addition, the Florida 
study developed a composite measure termed College Pathway that assessed early course 
completion as a composite of several variables. The Achieving the Dream study had rich 
information about developmental course completion. Developmental needs in reading, writing, 
and mathematics were tracked at three levels below college level coursework. Results were 
modeled using completion of developmental math, writing, and reading with a ‘B’ or better within 
three years. The pattern of results was mixed across these analyses. The Florida study took a 
more granular approach to developmental education and modeled it as a binary outcome 
representing took and failed a developmental course. 

For developmental math in the Achieving the Dream study, the Active and Collaborative 
Learning benchmark had a positive net effect when predicting course completion with a ‘B’ or 
better in coursework three levels below college level. No other benchmarks were positive 
predictors of developmental math course completions at any level, although students’ perceived 
Academic Gains had a positive net effect when predicting level 1 and level 2 developmental math 
course completions with a ‘B’ or better within three years. The Academic Preparation item cluster 
was also a significant bivariate predictor in the level 1 and level 2 developmental math courses 
and was a net predictor in level 2 developmental math. 

For developmental writing, engagement does not predict successful course completion 
with a ‘B’ or better two or more levels below college English; moreover, the Support for Learners 
benchmark and the related School Opinions item cluster were negative predictors of 
developmental English completion one level below college with a ‘B’ or better within three years. 
However, neither of these effects exhibited a significant bivariate relationship with the outcome, 
suggesting that one or more of the other variables in the model is producing a suppressor effect. 
Academic Preparation produced a significant bivariate relationship in both levels of 
developmental writing, and self-reported Academic Gains were also related to the completion of 
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developmental English level 2 with a ‘B’ or better. For both levels of developmental reading, the 
Student Effort benchmark had positive net and bivariate effects when predicting developmental 
reading course completions with a ‘B’ or better within three years. The Class Assignments and 
Technology Experiences item clusters were also positive predictors of completing developmental 
reading level 2 with a ‘B’ or better. 

Consistent with the Achieving the Dream results, CCSSE factors have relatively weak 
relationships with taking and passing developmental courses — both direct and after controls — in 
the Florida study. Academic Gains exhibited a significant net effect in the long cohort and Active 
and Collaborative Learning, Academic Preparation, and Class Assignments exhibited significant 
net effects in the short cohort. Thus, the effects that did emerge in the Florida study were the 
same effects that emerged in the Achieving the Dream developmental math models. 

Using completion of college algebra and college English with a ‘C’ or better within three 
years as outcome measures yielded mixed results for the Achieving the Dream study. The most 
promising results were for college algebra: the Active and Collaborative Learning benchmark had 
a positive “net effect” when predicting the completion of college algebra within three years. Two 
item clusters (Class Assignments and Collaborative Learning) were also positive predictors of 
completing college algebra with a ‘C’ or better, as was students’ perceived Academic Gains. 
There were no net effects for CCSSE benchmarks when predicting the completion of college 
English with a ‘C’ or better within three years. 

CCSSE constructs have weak relationships with taking and passing gatekeeper courses, 
both direct and after controls, according to the Florida validation study. Only Class Assignments 
showed significant net effects in predicting gatekeeper coursework performance for the short 
cohort group. In the cross-sectional file, grade-point performance for those who did take 
gatekeeper courses shows significant bivariate and net effects for most CCSSE “academic” item 
clusters including Student Effort, Academic Challenge, Class Assignments, and Academic 
Preparation. 

College Pathway Status was an intermediate completion measure defined as completing 
1 2 semester hours (or equivalent) of college credit. This measure showed bivariate and net 
effects on numerous factors. Significant bivariate and net effects on College Pathway Status 
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emerged for Active and Collaborative Learning, Support for Learners, Class Assignments, 
Collaborative Learning, Student Services, Academic Preparation, perceived Academic Gains, and 
perceived Vocational Goal Gains; and all measures except Exposure to Diversity and Information 
Technology exhibited significant bivariate relationships with this measure. 

Effects for developmental and gatekeeper course completion exhibited weaker 
relationships than most other outcome measures examined in the three studies. The Florida 
study examined the relationship between CCSSE factors and gatekeeper courses by modeling 
the outcome of “took and failed at least one gatekeeper course,” finding only occasional bivariate 
and net effects in relationships between these variables. The strongest effects found for this 
outcome measure were in the cross-sectional cohort. The Achieving the Dream study examined 
gatekeeper math and English course completion with a C or better within the first three years. 
While there were virtually no significant bivariate or net effects for English gatekeeper courses, 
there were several factors that exhibited significant bivariate relationships with college algebra, 
and net effects emerged for the Active and Collaborative Learning benchmark as well as Class 
Assignments, Collaborative Learning, and Academic Gains item clusters. The approach taken in 
the Achieving the Dream study separates math and English courses and finds that the strength of 
the relationship differs notably between these two outcomes. It would be useful to pursue this 
distinction in the Florida sample, and if a similar effect were observed, this would account for 
weaker effects found in that study where the effects for math courses were potentially diluted by 
effects from English courses. While the overall effects for developmental and gatekeeper 
courses were weak, the effects that did emerge were consistent. Academic Preparation and 
Gains in Academic outcomes emerged in a number of analyses as having positive relationships 
with developmental and gatekeeper courses, particularly math courses. The Student Effort 
benchmark and Class Assignments item cluster also emerged more than once in these analyses 
as being positively associated with developmental and gatekeeper course completion. It is 
notable that the composite variable, attainment of College Pathway Status, which should have a 
high overlap with developmental and gatekeeper courses, exhibited much stronger relationships 
than did the individual courses. It appears that among the detectable effects within 
developmental and gatekeeper course completion, measures of effort emerge as the strongest 
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predictors of course completion and this in turn results in higher levels of perceived Academic 
Gain. 

Persistence Measures 

Each of the studies examined persistence measures. First to second term persistence 
and first year to second year persistence were the most common measures. The Florida and 
Achieving the Dream datasets contained degree/certificate completion data that are considered 
here as well, although this variable could be considered an academic measure. 

The shortest term retention measure was within-term persistence, examined in the 
Achieving the Dream study by using Credit Completion Ratio as a measure of within-term 
persistence. Positive net effects for CCSSE benchmarks and item clusters were apparent when 
predicting Credit Completion Ratios within the same term CCSSE was administered - if students 
took the CCSSE in the spring of their first year. The same measure for students who took the 
CCSSE in the spring of their second year yielded no net effects. 

First to second term persistence was examined thoroughly in the Florida and FISS 
studies. Few CCSSE constructs are significantly related to next term persistence after controls 
are introduced in Florida’s long cohort validation study. Those net effects that emerged as 
significant are for item clusters that the literature suggests should be related to persistence — 
Collaborative Learning and Student Services. For the short cohort study, significant net effects 
on persistence to the next term emerged for a number of CCSSE constructs, including Active and 
Collaborative Learning and Support for Learners benchmarks and Faculty Interactions and 
Collaborative Learning item clusters, while virtually all CCSSE constructs exhibited significant 
bivariate correlations. Overall, 93 percent of the students in the HSS sample persisted from the 
first to second term. Thus, there was very little variance to predict in this outcome measure. 
Overall, in the HSS study, the Active and Collaborative Learning and Student-Faculty Interaction 
scales were the strongest predictors of first to second term persistence. 

First to second year persistence results were similar to those for first to second term 
persistence for the Florida long cohort validation study; Collaborative Learning and Student 
Services were significant net predictors of first to second year persistence. The Achieving the 
Dream study used fall-to-fall persistence as the outcome measure, yielding positive net effects for 
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the Active and Collaborative Learning benchmark, as well as three CCSSE item clusters 
(Collaborative Learning, Information Technology, and Use of Services) and students’ perceived 
Academic Gains. For the HSS study, the Active and Collaborative Learning benchmark, the 
Support for Learners benchmark, and Gains in Academics made significant contributions to the 
prediction of first to second year persistence. 

The persistence measures results share a strong consistency with other academic 
measures, such as GPA, Credit Completion Ratios, and Degree/Certificate Completion. Active 
and Collaborative Learning and related item clusters (i.e., Class Assignments and Collaborative 
Learning), as well as Gains in Academics, consistently exhibited significant bivariate relationships 
with GPA, Credit Completion Ratio, and Degree/Certificate Completion, as well as First to Second 
Term Completion. However, there was relatively weak support for the relationship between 
Academic Challenge and the Mental Activities item cluster for persistence measures, in contrast 
to GPA, Credit Completion Ratio, and Degree/Certificate Completion. Moreover, Support for 
Learners and the Student Services item cluster, as well as the Gains in Vocational Goals, 
consistently exhibited significant bivariate relationships with persistence measures. However, 
persistence measures showed no relationship, even occasionally exhibited a negative 
relationship to GPA and Credit Completion Ratios, and exhibited only a minor relationship to 
Degree/Certificate Completion. Thus, among the engagement factors that exhibited clear trends 
in the persistence measures, it appears that Support for Learners and use of Student Services 
are more important for persistence, but that Academic Challenge and Mental Activities have little 
relationship to persistence — in contrast to the consistently strong relationship that these 
measures have with GPA, Credit Completion Ratios, and Degree/Certificate Completion. 
Completion Measures 

The Florida and Achieving the Dream studies examined degree/certificate completion in 
multiple regression models. In addition, the Florida study explored an alternative measure, 
attainment of Transfer-Ready Status, a variable that was derived from the completion of a cluster 
of courses. This alternative measure is an important consideration in community colleges, as it 
has often been argued that degree completion is not an adequate performance measure for 
community colleges where students often have goals that do not include degree attainment. 
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CCSSE constructs, analyzed in Florida’s validation study, are significant bivariate and net 
predictors of overall associate degree completion, as well as degree completion within three 
years. Active and Collaborative Learning, Academic Challenge, and Support for Learners 
benchmarks consistently predicted degree/certificate completion at three years and at any point, 
and there was some support for Student-Faculty Interaction in the bivariate correlations. 

The net effects for engagement when predicting degree or certificate attainment within 
three years were very positive for the Achieving the Dream study. Three CCSSE benchmarks - 
Active and Collaborative Learning, Academic Challenge, and Student-Faculty Interaction - had 
positive net effects when predicting graduation. Three item clusters (Faculty Interaction, 
Collaborative Learning and Academic Preparation) were also positive predictors of graduation, as 
was students’ perceived Career Gains. 

The Florida study also created a transfer-ready variable that provided a direct alternative 
to degree completion. Transfer-ready students had completed 30 credits, passed or placed out of 
all developmental work, completed English Composition, a college-level math course, and one 
college-level course in each basic discipline cluster (science, social science, and humanities). 
Transfer-ready status was significantly correlated with all benchmarks, engagement item clusters 
and gain item clusters, with three exceptions that were all marginally correlated. Net effects 
emerged for Academic Challenge, Support for Learners, and all gain items. 

Comparing results across the Florida and Achieving the Dream studies produced 
consistent support for Active and Collaborative Learning and Academic Challenge in degree 
attainment. Additionally, Support for Learners consistently demonstrated significant effects for 
measures of degree completion and transfer-ready status, though results often differed between 
bivariate and net effects, indicating that this measure is impacted by control variables. Student 
Effort and Student-Faculty Interaction exhibited some effects, though the inconsistency of the 
results for these factors suggests a weak relationship with completion. 

Longevity Measures 

Analyses for longevity variables — that is, Number of Terms enrolled and Total Credit 
Hours Completed — were primarily examined through bivariate correlations. These measures 
were considered hybrids of academics and persistence. There was overwhelming consistency 
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across studies indicating that these measures were consistently correlated with engagement 
factors. 

Outcomes Based on Student Characteristics 

A number of student characteristics were investigated with regard to engagement. The 
purpose of such analyses was to determine if it is reasonable to expect that all students are 
equally engaged. Understanding the impact of student characteristics on engagement has 
important implications for institutional assessment: while it is reasonable to assume that 
institutions impact student engagement, it is also important to understand the extent to which 
students’ backgrounds may impact the way in which they engage and their levels of engagement. 

Race and Ethnicity. The HSS and Achieving the Dream studies conducted basic 
comparisons of race/ethnicity for engagement measures. In the Achieving the Dream study, 
black, non-Hispanic students were more engaged than white students on the Student Effort, 
Academic Challenge, and Support for Learners benchmarks, and Hispanic students were more 
engaged than white students on the Student Effort and Support for Learners benchmarks. The 
HSS study examined differences between Hispanic and non-Hispanic students on the five 
CCSSE benchmarks, item clusters, and gain items. Consistent with the benchmark analysis for 
the Achieving the Dream study, Hispanic students reported higher levels of Student Effort and 
Support for Learners. In addition, Hispanic students reported lower levels of Student-Faculty 
Interaction. Hispanic students reported significantly higher Academic, Personal Development, and 
Vocational Goals Gains. 

Effects for race/ethnicity in multivariate regression models were consistently diminished 
in the HSS and Achieving the Dream studies. All regression models for the Achieving the Dream 
study included controls for race and ethnicity (binary variables for black, Hispanic and white). In 
Achieving the Dream models, race/ethnicity was not statistically significant in the regression 
models. In those cases where race and ethnicity did impact the predictive power of engagement, 
the effects were as expected, given existing literature: black and/or Hispanic students were less 
likely to have a successful outcome and white students were more likely to have a successful 
outcome. In HSS hierarchical regression models in which self-reported Academic, Personal 
Development, and Vocational Goals were treated as outcome measures, the influence of 
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Hispanic status made small but statistically significant increases in total variance explained after 
considering the influence of student engagement items was considered. 

Immigrant Status. The HSS study took an in-depth look at students’ immigrant status. 
Differences in the levels of engagement, gain indices, and satisfaction between immigrant 
students and their non-immigrant peers were analyzed in the HSS study. International students 
reported being much more engaged than US-born students on four student engagement items. 
The group differences were greatest on Student Effort and Support for Learners. International 
students reported significantly higher Academic, Personal Development, and Vocational Goals 
Gains than did US-born peers. When the Hispanic status and immigrant status variables were 
combined to yield four groups, Non-Hispanic immigrants reported significantly higher levels of 
engagement on four of the five scales: Active and Collaborative Learning, Student Effort, 
Academic Challenge, and Student-Faculty Interaction. Overall, Non-Hispanic non-immigrants 
reported the least Academic, Personal Development, and Vocational Goals Gains. Hispanic and 
Non-Hispanic international students reported the most (and almost identical) Academic, Personal 
Development, and Vocational Goals Gains. Hispanic international students were more satisfied 
with their community college experience than were the other three groups, although all groups 
reported very positive community college experiences. 

Income. There was little reliable financial information available for the three studies. The 
Achieving the Dream study used two CCSSE items as proxies for low-income status - reliance on 
grants and scholarships and reliance on public assistance. Their analyses revealed that low- 
income students reported being more engaged than other students on four of the CCSSE 
benchmarks: Active and Collaborative Learning, Student Effort, Student-Faculty Interaction and 
Support for Learners. 

The examination of student characteristics indicates that there are differences based on 
student characteristics. Generally, it appears that groups that are traditionally disadvantaged 
have higher levels of engagement; this pattern is true for racial minorities, immigrants, and low- 
income students. The conditional effects for race/ethnicity reported in the Florida study were 
consistently sparse, and the effects that emerged as significant were inconsistent. While each 
interaction effect would take individual consideration to understand, the more notable fact was 
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that race/ethnicity did not appear to consistently interact with CCSSE measures. In combination 
with results demonstrating that minorities typically have higher levels of engagement, this pattern 
suggests that the strength of the relationship between engagement and putative outcome 
measures was not typically different to a large degree based on race/ethnicity. 

A Look by Benchmark 

Active and Collaborative Learning. Active and Collaborative Learning was perhaps the 
most consistent predictor of student success across studies and across measures. Active and 
Collaborative Learning consistently was correlated with the cumulative academic measures, 
Number of Terms Enrolled and Credit Hours Completed. However, it was not unique with regard 
to these measures, as all CCSSE benchmarks were correlated with these outcomes. The impact 
of Active and Collaborative Learning distinguishes itself in the academic and persistence outcome 
measures. Credit Completion Ratio and Degree Completion correlations were examined in the 
Achieving the Dream and Florida studies, and Active and Collaborative Learning was correlated 
with both measures in both studies. In addition, Credit Completion Ratio was correlated with 
GPA across all studies. The only other benchmark that exhibited this consistent pattern of 
positive correlations across all three studies was Academic Challenge. Active and Collaborative 
Learning was the only benchmark that was correlated with First to Second Term Persistence and 
First to Second Year Persistence across all three studies, though several other benchmarks 
showed strong patterns of consistency across the three studies. Thus, the support for Active and 
Collaborative Learning suggests that this benchmark measures processes that are important for 
all of the outcomes measured in the studies described herein. 

The pattern of results across the three studies is consistent with prior research. 
Educational practices, such as class discussions, cooperative learning, and student-generated 
questions and talking points used in classroom discussions have been linked with improved 
academic performance (Tsui, 2002; Connor-Greene, 2005). Specific practices, such as tutoring, 
have a positive impact on students’ academic performance (Yonhong, Hartman, Uribe, & 

Mencke, 2001). In addition to positive academic outcomes, active and collaborative engagement 
activities, such as class discussions, examination preparation, and higher order thinking activities 
influence social integration, institutional commitment, and students’ intent to return (Braxton, 
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Milem, & Sullivan, 2000). The results presented herein are generally consistent with previous 
work linking Active and Collaborative Learning with both academic and persistence measures. 

Student Effort. The results across models and studies suggest that the Student Effort 
benchmark is predictably related to retention measures and shows moderate predictability to 
academic measures. Number of Terms Enrolled and Credit Hours Completed were consistently 
correlated with Student Effort. The relationship between the Student Effort benchmark and 
academic measures was positive, though not completely consistent across studies. Student 
Effort exhibited the strongest consistency with GPA, where there were significant correlations 
across all three studies. The benchmark was correlated with Credit Completion Ratio in the 
Achieving the Dream study, but not in the Florida study, and was not correlated with 
Degree/Certificate Completion in either of these studies. Student Effort was correlated with First 
to Second Term Persistence and to First to Second Year Persistence in the Achieving the Dream 
and HSS studies, but not the Florida study. 

In sum, the Student Effort benchmark is a consistent predictor of persistence and 
provides mixed results for academic measures. These findings are consistent with previous 
research that has examined activities related to Student Effort such as amount of reading of 
course materials, level of note-taking, frequency of class attendance, and preparing multiple 
drafts of an assignment. Students participating in these activities have improved writing and 
revising skills (Carifio, Jackson, & Dagostino, 2001) and have increased exam performance 
(Williams & Clark, 2004). 

Academic Challenge. Academic Challenge was consistently associated with academic 
outcomes, while showing little evidence of being correlated with persistence measures. Number 
of Terms Enrolled, Credit Hours Completed, GPA, Credit Completion Ratio, and 
Degree/Certificate Completion were consistently correlated with Academic Challenge across all 
studies. However, Academic Challenge exhibited a correlation with First to Second Term or First 
to Second Year Persistence in only the HSS study. The pattern of results indicates that the 
Academic Challenge benchmark is indeed measuring behaviors that relate to academic 
outcomes. The outcomes in which Academic Challenge distinguishes itself are all related to 
academic success. 
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Considerable research exploring how Academic Challenge relates to student outcomes 
was seen in the literature. Students learn more when they are asked to tackle complex and 
compelling problems that invite them to develop an array of workable and innovative solutions 
(Kezar, Hirsch, & Burack, 2001). Use of unconventional, challenging assignments has been 
demonstrated to develop critical thinking skills (Herman, 2005). Gains in cognitive and 
communication skills are associated with both academic and co-curricular involvement (Huang & 
Chang, 2004). Thus, the results presented herein are consistent with previous empirical work 
examining the impact of Academic Challenge. 

Student-Faculty Interaction. The Student-Faculty Interaction benchmark results were 
positive, but the least consistent across the five benchmarks that were examined. Consistent 
with other benchmarks, it was correlated with Number of Terms Enrolled and Credit Hours 
Completed. However, results across academic and persistence measures were mixed. The 
Student-Faculty Interaction benchmark correlated with GPA in the Achieving the Dream and HSS 
studies, but not the Florida study; it correlated with credit completion ratio in the Achieving the 
Dream, but not the Florida study. In both the Achieving the Dream and the Florida studies, 
Student-Faculty Interactions correlated with Degree/Certificate Completion. Measures of 
persistence were inconsistent: Student-Faculty Interaction exhibited a correlation with First to 
Second Term Persistence in the Achieving the Dream and HSS studies and First to Second Year 
Persistence in only the HSS study. The measures that exhibited consistent relationships with 
Student-Faculty Interaction were Number of Terms Enrolled, Credit Hours Completed, and 
Degree/Certificate Completion. These three measures are arguably measuring both academic 
performance and persistence, in contrast to other measures, such as GPA, that could reasonably 
be considered primarily academic and term to term persistence, which could reasonably be 
considered a measure of persistence. Thus, the results indicate that Student-Faculty Interactions 
are impacting both academic and persistence outcomes. 

The link between Student-Faculty Interaction and positive academic achievement has 
support in the extant literature. Significant progress in improving student learning can be attained 
when students and faculty work collaboratively (Kezar et al., 2001 ). Frequent student interaction 
with faculty is a strong predictor of learning across all racial groups (Lundberg & Schreiner 2004). 
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Students value response formats that allow them to be active participants on feedback on written 
papers (Edgington, 2004). Wilson and Taylor (2001) linked professor immediacy to student 
motivation, projected grades, and evaluations of the instructor. Thus, the existing literature 
suggests that students value Student-Faculty Interaction and faculty feedback and that frequent 
interactions with faculty translate into improved learning. 

Support for Learners. The Support for Learners benchmark was consistently correlated 
with measures of persistence, but showed little evidence of being correlated with academic 
measures. Consistent with other benchmarks, the Support for Learners benchmark was 
correlated with Number of Terms Enrolled and Credit Hours Completed. There was not a single 
positive correlation between Support for Learners and GPA or Credit Completion Ratio across the 
three studies. The Florida study reported a correlation between Degree/Certificate Completion 
and the Support for Learners benchmark, while the Achieving the Dream study did not find this 
relationship. In contrast to the academic measures, there was good support for correlations 
between persistence measures and the Support for Learners benchmark. In both the HSS and 
the Florida study, the Support for Learners benchmark was correlated with First to Second Term 
Persistence and First to Second Year Persistence. The results that emerge from the Support for 
Learners benchmark analyses suggest that this benchmark has its greatest impact on 
persistence. The absence of a relationship with academic measures may indicate that to a large 
degree, students who report higher levels of Support for Learners are academically unprepared. 
Examination of item clusters shows that the Student Services item cluster is a strong predictor of 
persistence and degree completion but is virtually unrelated to academic measures and even 
exhibits a negative effect in the one instance that there is a significant effect in the GPA results. 
Use of student services is an important component of the Support for Learners benchmark, and 
when student services are isolated in the examination of the Student Services item cluster, use of 
student services provides an amplified version of the Support for Learners benchmark. This 
suggests that the student services items in the Support for Learners benchmark may drive this 
effect observed between the Support for Learners benchmark and Number of Terms Enrolled and 
Credit Hours Completed. Thus, use of student services provides support to maintain persistence 
but does not necessarily translate into higher academic performance. However, to the extent to 
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which use of services is compensatory for inadequate previous preparation, it is logically possible 
that there is an effect whereby student and academic support services raise performance to the 
level of better prepared students. 

These results support previous work that focuses on institutional practices promoting 
Support for Learners. Learning occurs best when students are in an environment in which they 
feel connected, cared for, and trusted (Kezar et al., 2001). Group interaction and support offer 
students the structure to integrate and engage in the educational process and provide a support 
structure that encourages retention (Wild & Ebbers, 2002). College mentors introduce students 
to their college community and help students develop a self-awareness that leads to a sense of 
agency and responsibility (Vivian, 2005). 

A Look by Gain Indicator 

Gains in Academics. The Gains in Academics item cluster was the item cluster that most 
consistently predicted student outcomes across studies and outcome measures. This gain item 
was consistently correlated with Number of Terms Enrolled, Total Credit Hours Completed, First 
to Second Term Persistence, and First to Second Year Persistence measures across all three 
studies. The Gains in Academics item cluster was also related to Credit Completion Ratio and 
Degree/Certificate Completion in the AtD and Florida studies. The pattern of results across the 
three studies is consistent with prior research. Academic integration has previously been 
demonstrated to be an important predictor of subsequent institutional commitment (Berger & 
Milem, 1999) and persistence (Blecher, 2006), and high perceptions of academic ability have a 
positive impact on student persistence (Miller, Greene, Montalvo, Ravindran, & Nichols, 1996). A 
study conducted by Taniguchi & Kaufman (2005) found that academic preparedness increases 
completion among nontraditional students, which highlights the significance of results from the 
HSS study that demonstrate that Hispanic and Non-Hispanic international students reported the 
most Academic Development, Personal Development, and Vocational Goals Gains. The reported 
associations between the Academic Gains item cluster supports existing reports that academic 
integration and academic ability facilitate student retention. 

Gains in Personal Development. Results across studies and outcome measures indicate 
that the Gains in Personal Development item cluster is most consistently related to longevity and 
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persistence measures. Across all three studies, the Personal Development Gains item was 
consistently correlated with Number or Terms Enrolled and First to Second Term Persistence, 
and exhibited more modest support for First to Second Year Persistence. This item cluster also 
showed patterns of consistent correlations with Total Credit Flours completed. The extant 
literature supports a positive association between Personal Development Gains relate to student 
outcomes. For example, an ethically principled campus climate has a positive effect on students’ 
academic achievement and willingness to remain in college (Gardiner, 1 998). A study conducted 
by Attinasi (1 989) found that the extent and nature of socialization while in college has an 
influence on freshmen Mexican American student persistence, which is a finding supported by the 
HSS study’s report that Flispanic and Non-Hispanic international students reported the largest 
Gains in personal development. Gains in personal development are likely tied to positive 
outcomes because students who are confident about regulating their own activities are more 
confident about mastering academic subjects and are more likely to attain higher academic 
performance (Zimmerman, Bandura, & Martinez-Pons, 1992; Joo, Bong, & Choi, 2000). Thus, 
the reported associations between the Personal Gains and persistence item cluster supports 
existing findings that ethically principled campuses and socialization facilitate persistence. 

Gains in Vocational Goals. When compared to other gain items, the Gains in Vocational 
Goals was the least consistent predictor of student outcomes across studies. Nevertheless, this 
gain item was consistently correlated with the cumulative academic measure, Number of Terms 
Enrolled across all three studies. Total Credit Flours Completed and both First to Second Term 
and First to Second Year Persistence showed consistent patterns of association with Gains in 
Vocational Goals and a less consistent, yet positive relationship between the Vocational Goal 
Gains indicator and Degree/Certificate Completion was observed. The link between perceived 
Vocational Goal Gains and student outcomes has been studied in the literature. Vocational 
training has a positive effect on educational attainment (Roksa, 2006). A recent study conducted 
on first semester freshmen found that students with defined job-related career goals made more 
positive persistence decisions than their peers without an identified career goal (Hull-Blanks, 
Kurpius, Befort, Sollenberger, Nicpon, & Huser, 2005). Research conducted by Sandler (2000) 
indicates that adult students’ decision to re-enroll is affected by their perceived vocational futures 
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and career expectations. Students’ perceived school-employer linkages and job placement 
significantly predict confidence in degree completion (Person & Rosenbaum, 2006). Thus, the 
reported associations between the Vocational Goals item cluster supports existing reports that 
career goals and perceptions about linkages between education and careers have positive impact 
on student persistence. 
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CONCLUSIONS AND IMPLICATIONS 



Results Confirm a Long Tradition of Research on Student Engagement 

The studies presented herein confirm a vast body of research on student engagement 
(Pascarella & Terenzini, 2005). Results support major theoretical perspectives such as Astin’s 
(1 985) theory of involvement, in which student learning occurs as a function of a student’s level of 
academic and social involvement with the institutional environment. Quality of Student Effort is a 
function of the opportunities that an institution offers and the extent to which students make use 
of those opportunities in their academic, intellectual, personal, and interpersonal experiences in 
Pace’s (1984) theory. Tinto’s (1993) model of student departure emphasizes the role of 
academic and social integration as processes that promote persistence. In spite of the 
voluminous work supporting these theoretical perspectives, the present studies fill a critical gap in 
the literature: validation of student integration and engagement models using community college 
students. Higher education research overwhelmingly under represents empirical work conducted 
using community college students (Cofers & Somers, 2000; Pascarella, 1997; Townsend et al., 
2004), and this gap is particularly salient in the engagement literature (Wortman & Napoli, 1 996). 

The paucity of empirical evidence linking student engagement to retention in community 
colleges is highlighted in a recent review of empirical literature (Braxton, Hirschy, & McClendon, 
2004). They examine thirteen testable propositions of Tinto’s (1975) model of student 
persistence and found that only student entry characteristics garner strong empirical support, 
although they do find modest empirical support for the relationship between academic integration 
and departure. They describe Tinto’s theory of student departure as undetermined and open to 
empirical treatment in two-year colleges. Of the propositions, only student entering 
characteristics has robust empirical support. The testable propositions in Tinto’s model that are 
most relevant to CCSR measures, social and academic integration, were not deemed to be well 
supported in the extant literature that examined community college samples. 

The broad conclusion that can be reached from the present studies is that the current 
lack of support for student integration and engagement models is due to a lack of data rather than 
a lack of applicability of student integration and engagement models. These studies demonstrate 
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that the broad measures of student engagement on the CCSR are predictive of outcomes 
measuring academic success and persistence in community colleges. 



The Outcome Measure Matters 

The breadth of the studies presented herein provides insight into the outcome measures 
that are most influenced by student engagement as well as providing new knowledge about 
specific relationships between engagement constructs and various outcome measures. In 
reviewing results across the three studies, we broadly classified outcomes as academic or 
persistence outcomes, in addition to a fair number of outcomes classified as hybrids. 

The academic outcomes were predictably impacted by the Academic Challenge and 
Active and Collaborative Learning benchmarks and had reasonable support from the Student 
Effort and Student-Faculty Interaction benchmarks. GPA was consistently related to higher levels 
of engagement in Active and Collaborative Learning, Student Effort, and Academic Challenge in 
addition to garnering strong support from Student-Faculty Interaction. Active and Collaborative 
Learning and Academic Challenge were the strongest predictors of Credit Completion Ratio. 

Thus, academic outcomes are most predictably related to the benchmarks that focus on activities 
directly related to coursework. 

Completion of individual courses and course grades appear to have relatively weak 
relationships to measures of student engagement, in contrast to broader measures. The item 
clusters that did emerge as having impact on individual course completions were academically 
oriented. While measures from individual courses appear to have the greatest paucity of 
relationships between engagement measures and outcomes, other analyses suggest that course 
completion and grades are related to engagement behaviors. Therefore, we speculate that 
individual courses are not sensitive to the impact of engagement as measured by the CCSR, 
rather than concluding that there is not an impact of student engagement on developmental and 
gatekeeper courses. The analysis of College Path in the Florida study, an outcome measure that 
represents the completion of 12 credit hours, provides the most direct support for this assertion, 
as this composite variable was broadly related to engagement measures. The College Path 
variable approximates the cumulative achievement of completing developmental and gatekeeper 
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courses, suggesting that broad measures better capture than do course-specific measures. 
Because the CCSR is about experiences at the college in general (across courses and 
experiences during an entire academic year), this result is not surprising and suggests that single 
course outcomes should not be tied to CCSR measures. Further investigations of single course 
outcomes should limit engagement data to engagement in the courses being examined. 

There were two direct persistence measures examined by all studies: First to Second 
Term Persistence and First to Second Year Persistence. In addition to the ubiquitous effects of 
Active and Collaborative Learning, Student Effort and Support for Learners were the most 
consistent predictors of persistence. These benchmarks include items regarding use of student 
services, and the consistent relationship between the Student Services item cluster and 
persistence supports the importance of student services in persistence. The Class Assignments 
item cluster is largely comprised of Student Effort items and supports the importance of effort as 
an engagement measure that predicts persistence. 

There were at least two measures that we considered hybrid measures of academics and 
persistence: Number of Terms Enrolled and Credits Hours Completed. These measures 
represent longevity as a persistence dimension and require accumulation of credits, an academic 
measure. These two measures were the most ubiquitously related to engagement items and 
Gains in Academics, Vocational Goals, and Personal Development. Given their breadth, they 
provide important validation for the CCSR as the CCSR is broadly construed to measure 
students’ overall experience at that college. 

It is clear that the choice of outcome is important in investigating the impact of student 
engagement behaviors. Aside from the nearly ubiquitous impact of Active and Collaborative 
Learning, CCSSE benchmarks appear to differentially impact outcomes. Academic Challenge 
predictably has the strongest impact on academic measures. Support for Learners has the 
greatest impact on persistence measures. The Student-Faculty Interaction and Student Effort 
benchmarks are not as easily classified as predicting academic or persistence measures, but did 
show good consistency within measures across studies. The general consistency within 
measures across studies exhibited by all benchmarks indicates that there are specific effects for 
specific domains of engagement practices and behaviors. Furthermore, null results between 
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engagement practices/behaviors and outcomes that are not necessarily related to these 
practices/behaviors reduce the possibility of a positive response bias among academically 
successful students were there global positive relationships between engagement and outcome 
measures. 



Context of Current Research 

While the studies presented herein make significant contributions to the literature on 
student engagement in community college settings, there are some important contextual 
considerations. First, administration of the CCSR to students in spring semesters undoubtedly 
has an impact on the sample that completes the CCSR. Second, the survey asks students to 
evaluate their entire experience at the college during the academic year in which the CCSR is 
administered. Both of these considerations have implications for the effect sizes observed in the 
present studies. Specifically, the spring administration produces a restriction of range, and 
students’ evaluation of their entire experience that year increases the signal-to-noise ratio. The 
reported effect sizes are generally small; however, when we consider the impact of the spring 
administration and of the fact that students are reporting on their entire college experience that 
year, we recognize that the effect size is undoubtedly reduced by these factors. However, the 
true power of the current studies is in the pervasiveness and consistency of effects across 
multiple studies. Further, these effects hold in spite of restriction of range and large signal-to- 
noise ratios. 

Spring administration of the CCSR undoubtedly limits the range of the variables that were 
examined in the studies presented herein, due to the fact that many students who begin college 
in the fall semester do not return. The impact on the range of student engagement factors is 
unknowable, as these students are not in classrooms where the survey is administered. 

However, the impact on the range of outcome measures is apparent: students who do not 
complete or persist past their first semester do not graduate, do not accumulate credit hours, and 
by definition, do not persist. Furthermore, students who persist for longer periods of time are 
more likely to attend during a semester that the class is sampled for CCSSE administration. 
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Thus, the range of outcomes and likely the range of engagement measures are limited to 
students who have, for the most part, survived to at least their second semester. 

The vast majority of survey questions ask students to evaluate their entire experience 
that year at the college where they took the survey. This essentially requires them to average 
their experiences across a number of courses that could potentially represent a wide range of 
experiences, thus increasing the signal-to-noise ratio. This strategy captures a snapshot of the 
typical student experience of students attending college during spring semesters. However, the 
cost of such breadth is that it does not capture heterogeneity within students and is essentially 
the average of a given student’s experience for each survey item. Outcomes, such as grades 
and course completions, are also heterogeneous within individuals. A more precise signal would 
capture the heterogeneity of levels of engagement in a putative behavior or putative cluster of 
behaviors as they relate to the heterogeneity of an outcome. While this is logistically 
overwhelming, we believe that the effect sizes obtained within would only be enhanced by 
reducing the signal-to-noise ratio, and purer measures of effect sizes would thus be enhanced. 
The value of detecting small effects between the average experience of students at an institution 
and their average outcomes is the promise that there is a more powerful signal in the combined 
distributions of those experiences and outcomes that underlie the detected effects presented 
herein. 

The general conclusion of the considerations presented herein is that the reported effect 
sizes are conservative measures of the true effect size of student engagement. While these 
considerations undoubtedly impact the effect sizes reported in these studies, both the spring 
administration and questions about students’ entire experiences at a college are by design. 

Spring administrations are an attempt to capture the experiences of students who have had time 
to experience the college. Questions about the entire experience at a college are intended to 
understand those experiences with the maximum breadth possible. Developing precise measures 
of effect size is the work of targeted experimental or quasi-experimental research. For purposes 
of the current investigation, small effect sizes are sufficient to demonstrate that effects hold 
despite factors that should only diminish them. The validation of the CCSR is derived from the 
pervasiveness of effects that present themselves even under inauspicious circumstances. 
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Validation of the CCSR as a Measure of Institutional Effectiveness 



Validating the CCSR as a measure of institutional effectiveness was the primary purpose 
of the studies presented herein, and the results broadly confirm that the behaviors and 
experiences measured by the instrument are positively related to student outcomes. The role of 
CCSSE measures in institutional effectiveness is the evaluation of processes. In an input, 
process, output framework (Ewell, 1998), processes are the most difficult components to 
measure. Inputs, such as test scores, demographics, and income are easily obtainable, as are 
output measures, such as graduation rates, course completion rates, and grades. 

The strategy of the present studies was to link engagement measures as process 
indicators to input and output measures, with primary emphasis on output measures. This link is 
critical to validating the use of the CCSR as an instrument for assessment and improvement of 
institutional effectiveness, as it illustrates the processes in terms of student behaviors and 
experiences that impact outcomes. While process indicators are the most difficult to measure, 
they also represent the student experiences that colleges have the greatest opportunity to impact. 
And while outcome measures typically are given primacy as evaluation measures, they are the 
product of inputs and processes; clearly, then, impacts on these measures occur as a function of 
inputs and processes. In community colleges, where open admissions are the norm, institutions 
have little impact on inputs; therefore, the greatest area of potential institutional influence is 
through institutional practices that comprise processes. 

While the focus of the present studies was on linking processes to output, there was 
considerable attention given to inputs. The studies repeatedly demonstrate that input 
characteristics, such as race, income, and academic ability impact process measures. While 
there is considerable validity that can be derived from bivariate relationships between processes 
and outcomes, it is important to understand the extent to which these relationships are affected 
by student characteristics. To a large extent, bivariate correlations held up in multivariate 
regression models; this suggests that the relationship between engagement and outcomes is 
above and beyond that which is explained by inputs. While many effects were diminished after 
controlling for inputs, the consistent persistence of engagement as a predictor of outcomes is a 
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reminder that while there may be input characteristics that predict engagement, engagement is 
fundamentally independent of input characteristics and malleable to institutional influence. 
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APPENDICES 



Appendix A: Florida Community College System Validation Study Results 

Full Cohort Results 



Table A1 



Outcome: Cumulative GPA 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.095 


.000 


.322 


1120 


.115 


.000 


1956 


Student Effort 


0.079 


.002 


.319 


1120 


.044 


.050 


1956 


Academic Challenge 


0.076 


.003 


.317 


1120 


.077 


.001 


1956 


Student-Faculty Interaction 


0.050 


.049 


.315 


1120 


.031 


.175 


1956 


Support for Learners 


0.053 


.039 


.313 


1117 


.017 


.460 


1953 


Faculty Interactions 


0.092 


.000 


.320 


1120 


.087 


.000 


1956 


Class Assignments 


0.282 


.010 


.316 


1120 


.024 


.282 


1955 


Exposure to Diversity 


0.027 


.385 


.313 


1117 


.067 


.003 


1951 


Collaborative Learning 


0.219 


.040 


.315 


1120 


.040 


.080 


1955 


Information Technology 


0.007 


.791 


.313 


1118 


.020 


.387 


1954 


Mental Activities 


0.085 


.001 


.319 


1120 


.094 


.000 


1956 


School Opinions 


0.063 


.013 


.317 


1110 


.040 


.075 


1943 


Student Services 


0.011 


.676 


.308 


1099 


-.088 


.000 


1923 


Academic Preparation 


0.064 


.013 


.320 


1113 


.075 


.001 


1946 


Gains in Academics 


0.112 


.000 


.326 


1111 


.066 


.003 


1943 


Gains in Personal Development 


0.060 


.022 


.317 


1108 


-.043 


.057 


1939 


Gains in Vocational Goals 


0.039 


.128 


.315 


1110 


.009 


.667 


1942 



Table A2 

Outcome: Credit Completion Ratio 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.054 


.050 


.166 


1120 


.070 


.002 


1956 


Student Effort 


0.047 


.097 


.166 


1120 


.006 


.780 


1956 


Academic Challenge 


0.051 


.067 


.166 


1120 


.070 


.002 


1956 


Student-Faculty Interaction 


-0.007 


.810 


.163 


1120 


-.004 


.846 


1956 


Support for Learners 


0.048 


.091 


.167 


1117 


.031 


.173 


1953 


Faculty Interactions 


0.004 


.899 


.163 


1120 


.009 


.677 


1956 


Class Assignments 


0.061 


.028 


.168 


1120 


.056 


.013 


1955 


Exposure to Diversity 


-0.023 


.401 


.156 


1117 


.008 


.709 


1951 


Collaborative Learning 


0.050 


.072 


.166 


1120 


.051 


.025 


1955 


Information Technology 


-0.018 


.524 


.163 


1118 


.012 


.597 


1954 


Mental Activities 


0.034 


.221 


.165 


1120 


.052 


.022 


1956 


School Opinions 


0.041 


.142 


.168 


1110 


.034 


.138 


1943 


Student Services 


0.053 


.071 


.162 


1099 


-.017 


.469 


1923 


Academic Preparation 


0.075 


.008 


.170 


1113 


.090 


.000 


1946 


Gains in Academics 


0.083 


.003 


.173 


1111 


.082 


.000 


1943 


Gains in Personal Development 


0.046 


.113 


.166 


1108 


-.006 


.799 


1939 


Gains in Vocational Goals 


0.046 


.107 


.169 


1110 


.040 


.079 


1942 
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Table A3 



Outcome: Percent Courses Completed with Grade of "C" or Better 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.064 


.019 


.197 


1121 


.087 


.000 


1958 


Student Effort 


0.055 


.047 


.197 


1121 


.008 


.736 


1958 


Academic Challenge 


0.065 


.018 


.197 


1121 


.076 


.001 


1958 


Student-Faculty Interaction 


-0.004 


.871 


.193 


1121 


-.004 


.870 


1958 


Support for Learners 


0.036 


.200 


.194 


1118 


.013 


.570 


1955 


Faculty Interactions 


0.018 


.525 


.193 


1121 


.022 


.321 


1958 


Class Assignments 


0.043 


.111 


.197 


1121 


.040 


.077 


1957 


Exposure to Diversity 


-0.018 


.515 


.192 


1118 


.017 


.462 


1953 


Collaborative Learning 


0.052 


.056 


.195 


1121 


.061 


.007 


1957 


Information Technology 


-0.025 


.363 


.193 


1119 


.003 


.907 


1956 


Mental Activities 


0.056 


.039 


.197 


1121 


.068 


.003 


1958 


School Opinions 


0.034 


.213 


.196 


1111 


.019 


.403 


1945 


Student Services 


0.051 


.075 


.193 


1100 


-.033 


.145 


1925 


Academic Preparation 


0.078 


.005 


.201 


1114 


.095 


.000 


1948 


Gains in Academics 


0.103 


.000 


.205 


1112 


.085 


.000 


1945 


Gains in Personal Development 


0.050 


.077 


.196 


1109 


-.022 


.337 


1941 


Gains in Vocational Goals 


0.044 


.111 


.198 


1111 


.019 


.393 


1944 



Table A4 

Outcome: Completed Associates Degree 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.091 


.001 


.143 


1120 


.116 


.000 


1956 


Student Effort 


0.135 


.031 


.282 


1120 


-.018 


.417 


1956 


Academic Challenge 


0.142 


.087 


.002 


1120 


.076 


.001 


1956 


Student-Faculty Interaction 


0.138 


.054 


.057 


1120 


.088 


.000 


1956 


Support for Learners 


0.083 


.004 


.142 


1117 


.029 


.201 


1953 


Faculty Interactions 


0.055 


.056 


.139 


1120 


.079 


.000 


1956 


Class Assignments 


0.045 


.112 


.138 


1120 


.050 


.028 


1955 


Exposure to Diversity 


-0.013 


.653 


.134 


1117 


.033 


.144 


1951 


Collaborative Learning 


0.092 


.001 


.142 


1120 


.116 


.000 


1955 


Information Technology 


0.089 


.002 


.142 


1118 


.107 


.000 


1954 


Mental Activities 


0.077 


.006 


.141 


1120 


.071 


.002 


1956 


School Opinions 


0.057 


.047 


.139 


1110 


.010 


.659 


1943 


Student Services 


0.073 


.014 


.139 


1099 


.000 


.999 


1923 


Academic Preparation 


0.053 


.056 


.139 


1113 


.079 


.001 


1946 


Gains in Academics 


0.112 


.000 


.153 


1111 


.082 


.000 


1943 


Gains in Personal Development 


0.070 


.017 


.140 


1108 


.014 


.531 


1939 


Gains in Vocational Goals 


0.047 


.105 


.143 


1110 


.013 


.567 


1942 
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Table A5 



Outcome: Completed Associates Degree within Three Years 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.064 


.028 


.085 


1120 


.095 


.000 


1956 




- 


.958 


.079 


1120 


-.008 


.710 


1956 


Student Effort 


0.002 














Academic Challenge 


0.068 


.020 


.084 


1120 


.068 


.003 


1956 


Student-Faculty Interaction 


0.042 


.152 


.082 


1120 


.073 


.001 


1956 


Support for Learners 


0.077 


.009 


.086 


1117 


.050 


.026 


1953 


Faculty Interactions 


0.054 


.067 


.084 


1120 


.071 


.002 


1956 


Class Assignments 


0.017 


.553 


.080 


1120 


.054 


.018 


1955 


Exposure to Diversity 


0.003 


.913 


.080 


1117 


.022 


.333 


1951 


Collaborative Learning 


0.059 


.044 


.083 


1120 


.090 


.000 


1955 


Information Technology 


0.047 


.110 


.075 


1118 


.061 


.007 


1954 


Mental Activities 


0.053 


.069 


.082 


1120 


.050 


.026 


1956 


School Opinions 


0.058 


.048 


.083 


1110 


.033 


.140 


1943 


Student Services 


0.039 


.207 


.082 


1099 


.010 


.649 


1923 


Academic Preparation 


0.060 


.045 


.084 


1113 


.083 


.000 


1946 


Gains in Academics 


0.112 


.000 


.096 


1111 


.079 


.001 


1943 


Gains in Personal Development 


0.057 


.059 


.082 


1108 


.015 


.515 


1939 


Gains in Vocational Goals 


0.064 


.032 


.088 


1110 


.321 


.000 


1942 



Table A6 

Outcome: Transfer-ready 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.039 


0.187 


0.028 


1120 


0.054 


0.017 


1956 


Student Effort 


0.034 


0.262 


0.030 


1120 


0.046 


0.044 


1956 


Academic Challenge 


0.082 


0.007 


0.032 


1120 


0.082 


0.000 


1956 


Student-Faculty Interaction 


0.052 


0.084 


0.029 


1120 


0.056 


0.013 


1956 


Support for Learners 


0.066 


0.032 


0.032 


1117 


0.060 


0.008 


1953 


Faculty Interactions 


0.042 


0.170 


0.028 


1120 


0.042 


0.060 


1956 


Class Assignments 


0.034 


0.254 


0.027 


1120 


0.057 


0.011 


1955 


Exposure to Diversity 


0.030 


0.313 


0.027 


1117 


0.041 


0.068 


1951 


Collaborative Learning 


0.029 


0.328 


0.027 


1120 


0.045 


0.046 


1955 


Information Technology 


0.036 


0.230 


0.027 


1118 


0.044 


0.050 


1954 


Mental Activities 


0.073 


0.014 


0.031 


1120 


0.069 


0.002 


1956 


School Opinions 


0.040 


0.191 


0.028 


1110 


0.044 


0.053 


1943 


Student Services 


0.057 


0.071 


0.036 


1099 


0.049 


0.033 


1923 


Academic Preparation 


0.072 


0.018 


0.031 


1113 


0.073 


0.001 


1946 


Gains in Academics 


0.143 


0.000 


0.046 


1111 


0.139 


0.000 


1943 


Gains in Personal Development 


0.087 


0.005 


0.033 


1108 


0.086 


0.000 


1939 


Gains in Vocational Goals 


0.069 


0.024 


0.032 


1110 


0.083 


0.000 


1942 
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Table A7 



Outcome: Persist Next Term 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.066 


.022 


.094 


1120 


.063 


.005 


1956 


Student Effort 


0.031 


.294 


.092 


1120 


.044 


.054 


1956 


Academic Challenge 


-0.005 


.875 


.089 


1120 


.025 


.279 


1956 


Student-Faculty Interaction 


-0.010 


.734 


.089 


1120 


.004 


.844 


1956 


Support for Learners 


0.037 


.208 


.091 


1117 


.052 


.022 


1953 


Faculty Interactions 


0.012 


.681 


.089 


1120 


.015 


.500 


1956 


Class Assignments 


0.352 


.005 


.096 


1120 


.044 


.051 


1955 


Exposure to Diversity 


0.019 


.503 


.090 


1117 


.008 


.710 


1951 


Collaborative Learning 


0.057 


.048 


.093 


1120 


.068 


.003 


1955 


Information Technology 


-0.067 


.020 


.094 


1118 


-.061 


.007 


1954 


Mental Activities 


-0.013 


.642 


.089 


1120 


.006 


.799 


1956 


School Opinions 


0.012 


.679 


.090 


1110 


.028 


.214 


1943 


Student Services 


0.079 


.009 


.094 


1099 


.082 


.000 


1923 


Academic Preparation 


0.227 


.062 


.092 


1113 


.038 


.092 


1946 


Gains in Academics 


0.001 


.971 


.090 


1111 


.051 


.025 


1943 


Gains in Personal Development 


0.001 


.969 


.088 


1108 


.048 


.036 


1939 


Gains in Vocational Goals 


0.035 


.242 


.091 


1110 


.060 


.008 


1942 



Table A8 

Outcome: Persist Next Year 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.100 


.001 


.094 


1120 


.085 


.000 


1956 


Student Effort 


0.034 


.249 


.084 


1120 


.029 


.202 


1956 


Academic Challenge 


0.008 


.776 


.083 


1120 


.025 


.278 


1956 


Student-Faculty Interaction 


0.030 


.304 


.084 


1120 


.041 


.072 


1956 


Support for Learners 


0.042 


.159 


.084 


1117 


.053 


.019 


1953 


Faculty Interactions 


0.031 


.298 


.084 


1120 


.040 


.074 


1956 


Class Assignments 


0.086 


.003 


.097 


1120 


.077 


.001 


1955 


Exposure to Diversity 


0.019 


.513 


.083 


1117 


-.006 


.788 


1951 


Collaborative Learning 


0.090 


.002 


.092 


1120 


.085 


.000 


1955 


Information Technology 


0.036 


.222 


.085 


1118 


.031 


.169 


1954 


Mental Activities 


-0.004 


.897 


.082 


1120 


.005 


.817 


1956 


School Opinions 


0.018 


.542 


.082 


1110 


.039 


.086 


1943 


Student Services 


0.063 


.038 


.084 


1099 


.043 


.057 


1923 


Academic Preparation 


0.254 


.037 


.085 


1113 


.044 


.050 


1946 


Gains in Academics 


0.021 


.465 


.082 


1111 


.057 


.012 


1943 


Gains in Personal Development 


0.039 


.192 


.084 


1108 


.033 


.150 


1939 


Gains in Vocational Goals 


-0.004 


.887 


.082 


1110 


.024 


.296 


1942 



106 




Table A9 



Outcome: Took and Failed at Least One Developmental Course 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.025 


.455 


.093 


856 


.003 


.916 


1150 


Student Effort 


-0.037 


.279 


.094 


856 


-.009 


.750 


1150 


Academic Challenge 


-0.047 


.162 


.094 


856 


-.015 


.610 


1150 


Student-Faculty Interaction 


0.031 


.360 


.093 


856 


.043 


.143 


1150 


Support for Learners 


-0.047 


.170 


.094 


854 


-.010 


.729 


1148 


Faculty Interactions 


0.015 


.661 


.093 


856 


.023 


.435 


1150 


Class Assignments 


-0.049 


.140 


.096 


856 


-.020 


.503 


1149 


Exposure to Diversity 


-0.016 


.630 


.093 


853 


-.022 


.460 


1145 


Collaborative Learning 


0.000 


.998 


.093 


856 


.036 


.224 


1150 


Information Technology 


0.281 


.074 


.095 


855 


-.011 


.718 


1149 


Mental Activities 


-0.028 


.406 


.093 


856 


.002 


.956 


1150 


School Opinions 


-0.064 


.057 


.098 


847 


-.028 


.338 


1139 


Student Services 


0.025 


.465 


.093 


841 


.056 


.062 


1131 


Academic Preparation 


-0.047 


.164 


.097 


849 


-.035 


.235 


1141 


Gains in Academics 


-0.070 


.036 


.099 


848 


-.062 


.037 


1140 


Gains in Personal Development 


-0.021 


.540 


.093 


845 


.038 


.201 


1137 


Gains in Vocational Goals 


0.013 


.710 


.094 


847 


.017 


.572 


1139 



Table A1 0 

Outcome: Took and Failed at Least One Gatekeeper Course 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.003 


.929 


.076 


1036 


.000 


.998 


1731 


Student Effort 


-0.038 


.215 


.077 


1036 


-.002 


.940 


1731 


Academic Challenge 


-0.012 


.685 


.077 


1036 


-.042 


.083 


1731 


Student-Faculty Interaction 


0.000 


1.000 


.076 


1036 


.035 


.142 


1731 


Support for Learners 


0.002 


.938 


.079 


1034 


.000 


.994 


1729 


Faculty Interactions 


-0.008 


.785 


.076 


1036 


.019 


.441 


1731 


Class Assignments 


-0.039 


.194 


.078 


1036 


-.007 


.760 


1731 


Exposure to Diversity 


0.043 


.152 


.080 


1033 


.016 


.516 


1727 


Collaborative Learning 


0.007 


.810 


.076 


1036 


.013 


.586 


1731 


Information Technology 


0.008 


.799 


.076 


1034 


.016 


.505 


1729 


Mental Activities 


-0.017 


.580 


.077 


1036 


-.036 


.139 


1731 


School Opinions 


0.004 


.900 


.077 


1027 


-.006 


.808 


1720 


Student Services 


-0.033 


.291 


.080 


1018 


.018 


.447 


1707 


Academic Preparation 


-0.035 


.266 


.077 


1029 


-.066 


.006 


1722 


Gains in Academics 


-0.013 


.661 


.075 


1028 


-.028 


.241 


1720 


Gains in Personal Development 


0.036 


.256 


.076 


1025 


.032 


.190 


1716 


Gains in Vocational Goals 


-0.061 


.048 


.079 


1027 


-.030 


0.218 


1719 
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Short Cohort Results 



Table A1 1 

Outcome: First Year GPA 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.063 


.010 


.136 


1476 


.065 


.001 


2656 


Student Effort 


0.051 


.042 


.134 


1476 


.047 


.016 


2656 


Academic Challenge 


0.037 


.138 


.133 


1476 


.065 


.001 


2656 


Student-Faculty Interaction 


-0.016 


.511 


.132 


1476 


-.013 


.503 


2656 


Support for Learners 


0.031 


.220 


.135 


1473 


.018 


.360 


2653 


Faculty Interactions 


-0.011 


.660 


.131 


1476 


.015 


.429 


2656 


Class Assignments 


0.080 


.001 


.141 


1476 


.067 


.001 


2655 


Exposure to Diversity 


-0.011 


.669 


.131 


1472 


.015 


.432 


2650 


Collaborative Learning 


0.042 


.089 


.134 


1476 


.021 


.289 


2655 


Information Technology 


0.000 


.993 


.133 


1474 


-.002 


.938 


2654 


Mental Activities 


0.039 


.108 


.133 


1476 


.067 


.001 


2656 


School Opinions 


0.036 


.150 


.134 


1465 


.029 


.131 


2638 


Student Services 


0.010 


.705 


.132 


1442 


-.270 


.175 


2594 


Academic Preparation 


0.012 


.643 


.133 


1465 


.040 


.041 


2641 


Gains in Academics 


0.052 


.035 


.134 


1465 


.027 


.163 


2637 


Gains in Personal Development 


0.008 


.746 


.132 


1462 


-.019 


.327 


2633 


Gains in Vocational Goals 


0.020 


.434 


.133 


1464 


.011 


.556 


2636 



Table A1 2 

Outcome: First Year Credit Completion Ratio 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.017 


.497 


.084 


1476 


.041 


.035 


2656 


Student Effort 


0.016 


.530 


.084 


1476 


.009 


.645 


2656 


Academic Challenge 


0.002 


.942 


.083 


1476 


.048 


.014 


2656 


Student-Faculty Interaction 


-0.026 


.307 


.085 


1476 


-.018 


.364 


2656 


Support for Learners 


0.060 


.010 


.091 


1473 


.016 


.400 


2653 


Faculty Interactions 


-0.035 


.170 


.085 


1476 


-.012 


.553 


2656 


Class Assignments 


0.058 


.020 


.088 


1476 


.066 


.001 


2655 


Exposure to Diversity 


-0.024 


.374 


.085 


1472 


-.011 


.584 


2650 


Collaborative Learning 


0.016 


.528 


.083 


1476 


.020 


.307 


2655 


Information Technology 


-0.020 


.438 


.084 


1474 


.003 


.875 


2654 


Mental Activities 


-0.013 


.594 


.084 


1476 


.034 


.083 


2656 


School Opinions 


0.056 


.001 


.090 


1465 


.015 


.445 


2638 


Student Services 


0.032 


.224 


.083 


1442 


-.004 


.852 


2584 


Academic Preparation 


0.027 


.290 


.084 


1468 


.047 


.016 


2641 


Gains in Academics 


-0.002 


.940 


.084 


1465 


.007 


.705 


2637 


Gains in Personal Development 


-0.017 


.523 


.085 


1462 


-.008 


.674 


2633 


Gains in Vocational Goals 


0.004 


.885 


.084 


1464 


.023 


.243 


2636 
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Table A1 3 



Outcome: First Year Percent Courses Completed with Grade of "C" or Better 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.035 


.130 


.218 


1477 


.020 


.308 


2658 


Student Effort 


0.035 


.140 


.218 


1477 


.019 


.317 


2658 


Academic Challenge 


0.046 


.051 


.219 


1477 


.038 


.048 


2658 


Student-Faculty Interaction 


-0.012 


.603 


.212 


1477 


-.034 


.083 


2658 


Support for Learners 


0.028 


.238 


.216 


1474 


-.001 


.944 


2655 


Faculty Interactions 


0.006 


.799 


.217 


1477 


-.005 


.793 


2658 


Class Assignments 


0.019 


.407 


.217 


1477 


.017 


.371 


2657 


Exposure to Diversity 


-0.016 


.491 


.217 


1473 


-.015 


.451 


2652 


Collaborative Learning 


0.025 


.295 


.217 


1477 


-.007 


.708 


2657 


Information Technology 


-0.024 


.313 


.215 


1475 


.004 


.826 


2656 


Mental Activities 


0.036 


.123 


.218 


1477 


.030 


.120 


2658 


School Opinions 


0.033 


.164 


.212 


1466 


.003 


.874 


2640 


Student Services 


0.029 


.240 


.219 


1433 


-.021 


.374 


2596 


Academic Preparation 


0.057 


.017 


.215 


1469 


.062 


.001 


2643 


Gains in Academics 


0.055 


.020 


.219 


1466 


.014 


.458 


2639 


Gains in Personal Development 


0.006 


.808 


.211 


1463 


-.048 


.014 


2635 


Gains in Vocational Goals 


0.019 


.431 


.211 


1465 


-.015 


.449 


2638 



Table A1 4 

Outcome: Persist Next Term 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.070 


.050 


.104 


1476 


.068 


.000 


2656 


Student Effort 


0.005 


.841 


.099 


1476 


.060 


.002 


2656 


Academic Challenge 


0.033 


.188 


.100 


1476 


.065 


.001 


2656 


Student-Faculty Interaction 


0.037 


.138 


.101 


1476 


.048 


.013 


2656 


Support for Learners 


0.061 


.017 


.105 


1473 


.081 


.000 


2653 


Faculty Interactions 


0.049 


.052 


.102 


1476 


.052 


.007 


2656 


Class Assignments 


0.045 


.072 


.103 


1476 


.083 


.000 


2655 


Exposure to Diversity 


0.040 


.115 


.100 


1472 


.030 


.118 


2650 


Collaborative Learning 


0.072 


.004 


.105 


1476 


.061 


.002 


2655 


Information Technology 


-0.036 


.151 


.101 


1474 


-.025 


.197 


2654 


Mental Activities 


0.033 


.191 


.101 


1476 


.048 


.014 


2656 


School Opinions 


0.062 


.016 


.104 


1465 


.062 


.001 


2638 


Student Services 


0.028 


.285 


.100 


1442 


.069 


.000 


2594 


Academic Preparation 


0.002 


.941 


.099 


1468 


.057 


.004 


2641 


Gains in Academics 


0.062 


.011 


.107 


1465 


.084 


.000 


2637 


Gains in Personal Development 


0.038 


.142 


.101 


1462 


.082 


.000 


2633 


Gains in Vocational Goals 


0.063 


.014 


.107 


1464 


.086 


.000 


2636 
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Table A1 5 



Outcome: College Path by End of First Year 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.078 


.000 


.307 


1476 


.100 


.000 


2656 


Student Effort 


0.040 


.078 


.303 


1476 


.095 


.000 


2656 


Academic Challenge 


0.029 


.193 


.302 


1476 


.083 


.000 


2656 


Student-Faculty Interaction 


0.030 


.181 


.302 


1476 


.070 


.000 


2656 


Support for Learners 


0.052 


.022 


.305 


1473 


.095 


.000 


2653 


Faculty Interactions 


0.036 


.105 


.302 


1476 


.072 


.000 


2656 


Class Assignments 


0.092 


.000 


.309 


1476 


.141 


.000 


2655 


Exposure to Diversity 


0.032 


.148 


.301 


1472 


.034 


.084 


2650 


Collaborative Learning 


0.059 


.007 


.304 


1476 


.085 


.000 


2655 


Information Technology 


-0.012 


.581 


.301 


1474 


-.009 


.639 


2654 


Mental Activities 


0.016 


.482 


.301 


1476 


.051 


.008 


2656 


School Opinions 


0.035 


.119 


.301 


1465 


.800 


.000 


2638 


Student Services 


0.303 


.039 


.090 


1442 


.095 


.000 


2594 


Academic Preparation 


0.052 


.022 


.303 


1468 


.107 


.000 


2641 


Gains in Academics 


0.065 


.004 


.305 


1465 


.101 


.000 


2637 


Gains in Personal Development 


0.025 


.285 


.303 


1462 


.069 


.000 


2633 


Gains in Vocational Goals 


0.060 


.008 


.304 


1464 


.085 


.000 


2636 



Table A1 6 

Outcome: Took and Failed at Least One Developmental Class 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.068 


.024 


.116 


996 


-.043 


.121 


1291 


Student Effort 


-0.050 


.110 


.114 


996 


-.025 


.367 


1291 


Academic Challenge 


-0.051 


.098 


.115 


996 


-.056 


.044 


1291 


Student-Faculty Interaction 


-0.010 


.747 


.112 


996 


.003 


.901 


1291 


Support for Learners 


-0.009 


.776 


.103 


994 


.011 


.705 


1289 


Faculty Interactions 


-0.019 


.528 


.112 


996 


-.013 


.630 


1291 


Class Assignments 


-0.089 


.003 


.120 


996 


-.059 


.034 


1290 


Exposure to Diversity 


-0.001 


.974 


.111 


992 


-.025 


.379 


1285 


Collaborative Learning 


-0.043 


.162 


.114 


996 


.001 


.974 


1291 


Information Technology 


-0.020 


.507 


.113 


995 


-.041 


.138 


1290 


Mental Activities 


-0.031 


.308 


.114 


996 


-.037 


.190 


1291 


School Opinions 


-0.011 


.722 


.112 


987 


-.001 


.962 


1281 


Student Services 


-0.004 


.896 


.109 


979 


.034 


.228 


1267 


Academic Preparation 


-0.071 


.023 


.116 


989 


-.074 


.008 


1283 


Gains in Academics 


-0.042 


.169 


.114 


988 


-.036 


.198 


1282 


Gains in Personal Development 


-0.008 


.800 


.113 


985 


.033 


.243 


1279 


Gains in Vocational Goals 


0.027 


.381 


.113 


987 


.041 


.146 


1281 
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Table A1 7 



Outcome: Took and Failed at Least One Gatekeeper Class 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.021 


.520 


.050 


918 


-.026 


.281 


1682 


Student Effort 


-0.049 


.141 


.051 


918 


-.039 


.114 


1682 


Academic Challenge 


0.015 


.645 


.050 


918 


-.053 


.029 


1682 


Student-Faculty Interaction 


-0.009 


.787 


.050 


918 


.004 


.878 


1682 


Support for Learners 


-0.004 


.903 


.052 


916 


-.025 


.303 


1680 


Faculty Interactions 


-0.010 


.757 


.050 


918 


-.001 


.956 


1682 


Class Assignments 


-0.080 


.014 


.056 


918 


-.064 


.009 


1682 


Exposure to Diversity 


-0.037 


.260 


.053 


914 


.008 


.737 


1677 


Collaborative Learning 


0.009 


.780 


.050 


918 


-.002 


.933 


1682 


Information Technology 


-0.061 


.018 


.055 


916 


-.022 


.365 


1680 


Mental Activities 


0.025 


.445 


.050 


918 


-.038 


.115 


1682 


School Opinions 


0.009 


.783 


.050 


910 


-.022 


.378 


1671 


Student Services 


-0.024 


.474 


.051 


898 


-.001 


.968 


1649 


Academic Preparation 


-0.048 


.158 


.053 


912 


-.080 


.001 


1673 


Gains in Academics 


-0.038 


.251 


.050 


910 


-.034 


.159 


1670 


Gains in Personal Development 


0.002 


.961 


.050 


907 


-.007 


.784 


1666 


Gains in Vocational Goals 


-0.028 


.401 


.051 


909 


-.027 


.272 


1669 



Cross Sectional Performance File Results 

Table A1 8 

Outcome: Three-Term GPA 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.095 


.000 


.121 


2051 


.104 


.000 


3176 


Student Effort 


0.094 


.000 


.121 


2054 


.072 


.000 


3180 


Academic Challenge 


0.080 


.000 


.120 


2053 


.069 


.000 


3179 


Student-Faculty Interaction 


0.052 


.014 


.116 


2045 


.048 


.007 


3160 


Support for Learners 


-0.002 


.940 


.113 


2034 


-.025 


.162 


3146 


Faculty Interactions 


0.075 


.000 


.118 


2052 


.097 


.000 


3175 


Class Assignments 


0.067 


.002 


.122 


1982 


.053 


.004 


3051 


Exposure to Diversity 


0.002 


.935 


.113 


1995 


.005 


.803 


3085 


Collaborative Learning 


0.036 


.096 


.116 


1890 


.023 


.216 


2929 


Information Technology 


0.036 


.101 


.112 


1941 


.015 


.398 


2992 


Mental Activities 


0.094 


.000 


.123 


2047 


.095 


.000 


3167 


School Opinions 


0.012 


.571 


.113 


2030 


-.017 


.331 


3146 


Student Services 


-0.007 


.753 


.115 


1801 


-.036 


.062 


2754 


Academic Preparation 


0.069 


.002 


.120 


2037 


.052 


.003 


3156 


Gains in Academics 


0.069 


.001 


.119 


2035 


.028 


.113 


3155 


Gains in Personal Development 


-0.030 


.164 


.114 


2034 


-.059 


.001 


3154 


Gains in Vocational Goals 


0.015 


.495 


.114 


2035 


.002 


.918 


3156 
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Table A1 9 



Outcome: Three-Term Credit Completion Ratio 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.087 


.000 


.034 


2163 


.056 


.001 


3504 


Student Effort 


0.094 


.000 


.037 


2166 


.033 


.051 


3505 


Academic Challenge 


0.067 


.002 


.037 


2163 


.050 


.030 


3506 


Student-Faculty Interaction 


0.134 


.029 


.028 


2155 


.002 


.914 


3485 


Support for Learners 


0.019 


.276 


.026 


2146 


-.012 


.467 


3467 


Faculty Interactions 


0.173 


.006 


.030 


2163 


.005 


.758 


3502 


Class Assignments 


0.081 


.000 


.035 


2085 


.054 


.002 


3359 


Exposure to Diversity 


0.025 


.250 


.026 


2102 


.039 


.002 


3401 


Collaborative Learning 


0.056 


.012 


.030 


1992 


.018 


.315 


3235 


Information Technology 


0.051 


.022 


.025 


2043 


.032 


.068 


3291 


Mental Activities 


0.053 


.013 


.036 


2158 


.041 


.016 


3493 


School Opinions 


0.024 


.262 


.027 


2139 


.006 


.716 


3467 


Student Services 


0.048 


.040 


.027 


1898 


-.017 


.348 


3018 


Academic Preparation 


0.083 


.000 


.032 


2148 


.070 


.000 


3482 


Gains in Academics 


0.089 


.000 


.037 


2146 


.055 


.001 


3481 


Gains in Personal Development 


0.021 


.337 


.028 


2145 


-.008 


.637 


3480 


Gains in Vocational Goals 


0.022 


.305 


.030 


2146 


.007 


.660 


3482 



Table A20 

Outcome: Percent Courses with A-C Grades 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.098 


.000 


.072 


2128 


.094 


.000 


3291 


Student Effort 


0.104 


.000 


.073 


2131 


.066 


.000 


3295 


Academic Challenge 


0.091 


.000 


.075 


2130 


.062 


.000 


3294 


Student-Faculty Interaction 


0.038 


.075 


.063 


2120 


.023 


.192 


3273 


Support for Learners 


-0.007 


.734 


.060 


2112 


-.046 


.090 


3261 


Faculty Interactions 


0.051 


.016 


.066 


2128 


.055 


.002 


3289 


Class Assignments 


0.097 


.000 


.072 


2053 


.083 


.000 


3155 


Exposure to Diversity 


0.034 


.118 


.063 


2067 


.007 


.676 


3192 


Collaborative Learning 


0.033 


.132 


.061 


1962 


.024 


.178 


3037 


Information Technology 


0.061 


.005 


.058 


2011 


.033 


.063 


3097 


Mental Activities 


0.081 


.000 


.074 


2124 


.065 


.000 


3282 


School Opinions 


0.010 


.654 


.061 


2104 


-.026 


.141 


3257 


Student Services 


0.026 


.268 


.062 


1868 


-.017 


.360 


2849 


Academic Preparation 


0.102 


.000 


.073 


2113 


.071 


.000 


3270 


Gains in Academics 


0.093 


.000 


.072 


2111 


.053 


.002 


3269 


Gains in Personal Development 


0.000 


.986 


.062 


2110 


-.041 


.089 


3268 


Gains in Vocational Goals 


0.033 


.121 


.064 


2111 


.009 


.598 


3270 
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Table A21 



Outcome: Grade Points in Gatekeeper Course 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.074 


.051 


.037 


641 


.800 


.014 


641 


Student Effort 


0.155 


.000 


.055 


642 


.138 


.000 


642 


Academic Challenge 


0.092 


.020 


.049 


642 


.100 


.002 


642 


Student-Faculty Interaction 


0.049 


.214 


.034 


640 


.033 


.302 


640 


Support for Learners 


0.005 


.906 


.032 


637 


.006 


.858 


637 


Faculty Interactions 


0.062 


.115 


.035 


642 


.062 


.056 


642 


Class Assignments 


0.207 


.000 


.074 


624 


.205 


.000 


624 


Exposure to Diversity 


0.003 


.933 


.037 


623 


.008 


.816 


623 


Collaborative Learning 


0.023 


.578 


.034 


586 


.023 


.488 


586 


Information Technology 


0.059 


.161 


.035 


599 


.022 


.515 


599 


Mental Activities 


0.077 


.053 


.042 


638 


.078 


.016 


638 


School Opinions 


0.021 


.602 


.032 


634 


.010 


.761 


634 


Student Services 


0.077 


.078 


.036 


551 


.036 


.297 


551 


Academic Preparation 


0.092 


.026 


.045 


638 


.120 


.000 


638 


Gains in Academics 


0.115 


.004 


.046 


637 


.073 


.025 


637 


Gains in Personal Development 


0.011 


.790 


.039 


637 


.035 


.286 


637 


Gains in Vocational Goals 


0.005 


.910 


.036 


637 


.035 


.280 


637 
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Appendix B: Achieving the Dream Validation Study Results 



Table B1 

Outcome: College Algebra Completion (C or better) by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


1.674 


.001 


.067 


660 


.128 


.001 


660 


Student Effort 


0.976 


.086 


.052 


660 


.065 


.094 


660 


Academic Challenge 


0.546 


.292 


.049 


660 


.047 


.233 


660 


Student-Faculty Interaction 


0.482 


.293 


.049 


660 


.047 


.230 


660 


Support for Learners 


0.498 


.213 


.049 


658 


.039 


.316 


658 


Faculty Interactions 


0.772 


.111 


.051 


660 


.067 


.086 


660 


Class Assignments 


0.909 


.016 


.058 


660 


.084 


.031 


660 


Exposure to Diversity 


0.374 


.230 


.049 


660 


.049 


.206 


660 


Collaborative Learning 


1.265 


.008 


.061 


660 


.101 


.009 


660 


Information Technology 


0.234 


.431 


.048 


660 


.037 


.345 


660 


Mental Activities 


0.352 


.384 


.048 


660 


.036 


.361 


660 


School Opinions 


0.419 


.274 


.047 


654 


.036 


.362 


654 


Student Services 


0.120 


.744 


.046 


647 


-.001 


.972 


647 


Academic Preparation 


1.036 


.069 


.052 


655 


.090 


.022 


655 


Gains in Academics 


0.386 


.004 


.062 


654 


.097 


.013 


654 


Gains in Personal Development 


0.096 


.368 


.047 


654 


.020 


.614 


654 


Gains in Vocational Goals 


0.095 


.349 


.047 


654 


.028 


.482 


654 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Developmental Math 
Placement Level, Sum of Risk Factors 



Table B2 

Outcome: College English Completion (C or better) by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.698 


.080 


.040 


1097 


-.049 


.103 


1097 


Student Effort 


-0.580 


.168 


.038 


1097 


-.037 


.218 


1097 


Academic Challenge 


-0.369 


.342 


.037 


1097 


-.028 


.361 


1097 


Student-Faculty Interaction 


-0.421 


.231 


.037 


1096 


-.030 


.323 


1096 


Support for Learners 


-0.095 


.753 


.036 


1097 


.005 


.862 


1097 


Faculty Interactions 


-0.493 


.176 


.038 


1097 


-.032 


.290 


1097 


Class Assignments 


0.013 


.962 


.036 


1097 


-.004 


.905 


1097 


Exposure to Diversity 


-0.231 


.332 


.037 


1097 


-.028 


.356 


1097 


Collaborative Learning 


-0.497 


.169 


.038 


1096 


-.036 


.233 


1096 


Information Technology 


-0.413 


.069 


.039 


1096 


-.064 


.034 


1096 


Mental Activities 


-0.404 


.178 


.038 


1096 


-.038 


.203 


1096 


School Opinions 


-0.047 


.870 


.037 


1088 


.016 


.589 


1088 


Student Services 


0.006 


.983 


.036 


1075 


.005 


.867 


1075 


Academic Preparation 


-0.529 


.214 


.039 


1088 


-.045 


.140 


1088 


Gains in Academics 


-0.063 


.522 


.037 


1088 


-.005 


.864 


1088 


Gains in Personal Development 


-0.013 


.868 


.037 


1088 


.019 


.531 


1088 


Gains in Vocational Goals 


-0.098 


.199 


.037 


1088 


-.015 


.617 


1088 



NOTE: Logistic regression model (R 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Developmental 
Reading Placement Level, Developmental Writing Placement Level, Sum of Risk Factors 
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Table B3 



Outcome: Developmental Math Completion, Level 1 (B or better) by Year 3 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff 


Sig. 


N 


Active and Collaborative Learning 


0.616 


.274 


.063 


491 


.058 


.155 


606 


Student Effort 


0.791 


.217 


.064 


491 


.096 


.019 


606 


Academic Challenge 


0.740 


.190 


.064 


491 


.079 


.051 


606 


Student-Faculty Interaction 


0.651 


.207 


.064 


490 


.075 


.067 


605 


Support for Learners 


-0.112 


.797 


.060 


489 


-.014 


.731 


604 


Faculty Interactions 


0.721 


.189 


.064 


491 


.081 


.045 


606 


Class Assignments 


0.217 


.591 


.060 


491 


.042 


.296 


606 


Exposure to Diversity 


0.296 


.393 


.062 


491 


.050 


.218 


606 


Collaborative Learning 


0.447 


.386 


.061 


490 


.034 


.398 


605 


Information Technology 


0.307 


.351 


.062 


490 


.062 


.130 


605 


Mental Activities 


0.598 


.177 


.064 


490 


.074 


.070 


605 


School Opinions 


-0.225 


.595 


.061 


486 


-.016 


.691 


601 


Student Services 


0.181 


.648 


.065 


477 


.017 


.680 


591 


Academic Preparation 


1.075 


.088 


.069 


487 


.094 


.021 


602 


Gains in Academics 


0.297 


.049 


.071 


486 


.113 


.006 


601 


Gains in Personal Development 


-0.054 


.652 


.061 


486 


-.041 


.318 


601 


Gains in Vocational Goals 


-0.021 


.854 


.060 


486 


-.024 


.560 


601 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 



Table B4 

Outcome: Developmental Math Completion, Level 2 (B or better) by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


1.060 


.059 


.073 


523 


.036 


.386 


582 


Student Effort 


0.574 


.344 


.067 


523 


.048 


.248 


582 


Academic Challenge 


0.949 


.080 


.072 


523 


.075 


.071 


582 


Student-Faculty Interaction 


0.022 


.964 


.066 


522 


-.022 


.593 


581 


Support for Learners 


0.555 


.179 


.069 


522 


.059 


.158 


581 


Faculty Interactions 


0.230 


.654 


.065 


523 


-.011 


.787 


582 


Class Assignments 


0.595 


.131 


.070 


523 


.017 


.685 


582 


Exposure to Diversity 


0.181 


.590 


.065 


523 


.003 


.943 


582 


Collaborative Learning 


0.716 


.161 


.071 


522 


.035 


.394 


581 


Information Technology 


0.185 


.573 


.067 


522 


.018 


.668 


581 


Mental Activities 


0.424 


.305 


.069 


522 


.044 


.284 


581 


School Opinions 


0.766 


.061 


.072 


516 


.083 


.046 


575 


Student Services 


0.263 


.490 


.068 


509 


.030 


.472 


566 


Academic Preparation 


1.346 


.038 


.074 


517 


.087 


.038 


576 


Gains in Academics 


0.559 


.000 


.099 


516 


.149 


.000 


575 


Gains in Personal Development 


0.048 


.681 


.064 


516 


-.003 


.937 


575 


Gains in Vocational Goals 


0.106 


.334 


.066 


516 


.024 


.561 


575 



NOTE: Logistic regression model (R 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 
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Table B5 



Outcome: Developmental Math Completion, Level 3 (B or better) by Year 3 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff 


Sig. 


N 


Active and Collaborative Learning 


2.392 


.000 


.162 


418 


.121 


.011 


446 


Student Effort 


0.114 


.868 


.126 


418 


.002 


.970 


446 


Academic Challenge 


0.551 


.386 


.128 


418 


.026 


.591 


446 


Student-Faculty Interaction 


0.195 


.733 


.125 


418 


.005 


.920 


446 


Support for Learners 


-0.368 


.449 


.128 


417 


-.047 


.324 


445 


Faculty Interactions 


0.621 


.300 


.129 


418 


.046 


.334 


446 


Class Assignments 


1.220 


.009 


.146 


418 


.073 


.123 


446 


Exposure to Diversity 


0.202 


.599 


.127 


418 


.024 


.617 


446 


Collaborative Learning 


1.651 


.006 


.147 


417 


.072 


.131 


445 


Information Technology 


0.729 


.054 


.135 


417 


.071 


.135 


445 


Mental Activities 


0.316 


.509 


.126 


417 


.007 


.890 


445 


School Opinions 


-0.371 


.440 


.129 


414 


-.047 


.327 


442 


Student Services 


-0.418 


.348 


.133 


405 


-.063 


.191 


433 


Academic Preparation 


1.168 


.121 


.134 


414 


.096 


.044 


442 


Gains in Academics 


0.220 


.174 


.133 


414 


.066 


.168 


442 


Gains in Personal Development 


-0.141 


.300 


.130 


414 


-.064 


.180 


442 


Gains in Vocational Goals 


-0.163 


.214 


.132 


414 


-.061 


.202 


442 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 



Table B6 

Outcome: Developmental English Completion, Level 1 (B or better) by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


-0.137 


.843 


.146 


387 


-.003 


.952 


457 


Student Effort 


0.241 


.751 


.146 


387 


.045 


.336 


457 


Academic Challenge 


0.258 


.715 


.146 


387 


.037 


.424 


457 


Student-Faculty Interaction 


-0.652 


.239 


.151 


386 


-.043 


.363 


456 


Support for Learners 


-1 .484 


.005 


.170 


385 


-.087 


.062 


455 


Faculty Interactions 


-0.646 


.271 


.149 


387 


-.035 


.454 


457 


Class Assignments 


1.004 


.062 


.157 


387 


.073 


.117 


457 


Exposure to Diversity 


-0.425 


.315 


.149 


387 


-.036 


.443 


457 


Collaborative Learning 


-0.075 


.899 


.147 


386 


-.019 


.690 


456 


Information Technology 


0.228 


.548 


.148 


386 


.059 


.210 


456 


Mental Activities 


0.033 


.951 


.147 


386 


.014 


.763 


456 


School Opinions 


-1.150 


.021 


.164 


381 


-.066 


.162 


451 


Student Services 


-0.557 


.274 


.155 


376 


-.043 


.365 


445 


Academic Preparation 


1.136 


.161 


.152 


382 


.113 


.016 


452 


Gains in Academics 


0.130 


.460 


.148 


381 


.047 


.322 


451 


Gains in Personal Development 


-0.221 


.152 


.153 


381 


-.038 


.419 


451 


Gains in Vocational Goals 


-0.074 


.603 


.147 


381 


-.023 


.621 


451 



NOTE: Logistic regression model (R 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 
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Table B7 



Outcome: Developmental English Completion, Level 2 or Lower (B or better) by Year 3 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff 


Sig. 


N 


Active and Collaborative Learning 


-0.854 


.388 


.139 


172 


-.005 


.944 


199 


Student Effort 


-0.020 


.986 


.133 


172 


.044 


.539 


199 


Academic Challenge 


0.690 


.510 


.137 


172 


.087 


.222 


199 


Student-Faculty Interaction 


-0.716 


.414 


.138 


171 


.006 


.932 


198 


Support for Learners 


-0.614 


.421 


.139 


171 


-.075 


.290 


198 


Faculty Interactions 


-1.109 


.226 


.144 


172 


-.007 


.920 


199 


Class Assignments 


0.473 


.544 


.136 


172 


.061 


.394 


199 


Exposure to Diversity 


0.670 


.323 


.141 


172 


.078 


.272 


199 


Collaborative Learning 


-0.329 


.687 


.134 


171 


-.002 


.979 


198 


Information Technology 


0.409 


.512 


.136 


171 


.087 


.223 


198 


Mental Activities 


0.093 


.908 


.133 


171 


.032 


.658 


198 


School Opinions 


-0.353 


.628 


.156 


170 


-.061 


.396 


197 


Student Services 


0.384 


.648 


.129 


160 


.037 


.619 


186 


Academic Preparation 


1.259 


.260 


.163 


170 


.152 


.033 


197 


Gains in Academics 


0.349 


.219 


.165 


170 


.156 


.028 


197 


Gains in Personal Development 


0.125 


.615 


.156 


170 


.066 


.354 


197 


Gains in Vocational Goals 


0.156 


.480 


.158 


170 


.081 


.260 


197 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 



Table B8 

Outcome: Developmental Reading Completion, Level 1 (B or better) by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.176 


.807 


.059 


347 


.008 


.872 


399 


Student Effort 


1.297 


.124 


.068 


347 


.105 


.036 


399 


Academic Challenge 


1.144 


.134 


.068 


347 


.095 


.058 


399 


Student-Faculty Interaction 


-0.202 


.749 


.059 


347 


-.007 


.891 


399 


Support for Learners 


-0.699 


.207 


.066 


346 


-.066 


.191 


398 


Faculty Interactions 


-0.141 


.831 


.059 


347 


.002 


.971 


399 


Class Assignments 


1.550 


.004 


.093 


347 


.139 


.005 


399 


Exposure to Diversity 


-0.233 


.622 


.060 


347 


-.022 


.656 


399 


Collaborative Learning 


-0.302 


.633 


.060 


347 


-.044 


.382 


399 


Information Technology 


0.574 


.176 


.066 


347 


.086 


.085 


399 


Mental Activities 


0.786 


.164 


.067 


347 


.082 


.102 


399 


School Opinions 


-0.625 


.241 


.066 


343 


-.058 


.250 


395 


Student Services 


-0.079 


.880 


.062 


339 


.005 


.927 


391 


Academic Preparation 


0.722 


.402 


.063 


343 


.087 


.085 


395 


Gains in Academics 


-0.070 


.723 


.061 


343 


.025 


.623 


395 


Gains in Personal Development 


-0.200 


.217 


.067 


343 


-.072 


.154 


395 


Gains in Vocational Goals 


-0.097 


.523 


.062 


343 


-.041 


.412 


395 



Data Sources: Achieving the Dream Database (July 6, 2006) and CCSSE (2003, 2004, 2005) 
NOTE: Logistic regression model (R 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 
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Table B9 



Outcome: Developmental Reading Completion, Level 2 (B or better) by Year 3 



CCSSE Predictor 




Regression 




Correlation 




Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.446 


.687 


.100 


154 


.077 


.302 


182 


Student Effort 


2.708 


.041 


.134 


154 


.163 


.028 


182 


Academic Challenge 


-0.069 


.947 


.099 


154 


.056 


.454 


182 


Student-Faculty Interaction 


0.677 


.520 


.102 


154 


.089 


.233 


182 


Support for Learners 


-0.935 


.261 


.109 


154 


-.115 


.122 


181 


Faculty Interactions 


0.227 


.837 


.099 


154 


.053 


.473 


182 


Class Assignments 


1.867 


.031 


.138 


154 


.154 


.038 


182 


Exposure to Diversity 


0.463 


.565 


.101 


154 


.053 


.477 


182 


Collaborative Learning 


0.073 


.937 


.099 


154 


.029 


.694 


182 


Information Technology 


1.806 


.007 


.161 


154 


.249 


.001 


182 


Mental Activities 


0.456 


.555 


.102 


154 


.086 


.250 


182 


School Opinions 


-0.881 


.271 


.109 


154 


-.116 


.120 


181 


Student Services 


1.394 


.101 


.130 


145 


.102 


.183 


172 


Academic Preparation 


-1.304 


.271 


.109 


154 


-.020 


.786 


182 


Gains in Academics 


0.096 


.731 


.100 


154 


.032 


.673 


181 


Gains in Personal Development 


0.188 


.467 


.103 


154 


.006 


.940 


181 


Gains in Vocational Goals 


0.017 


.943 


.099 


154 


-.056 


.452 


181 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Sum of Risk Factors 



Table BIO 

Outcome: Cumulative GPA (after two years) 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff 


Sig. 


N 


Active and Collaborative Learning 


0.820 


.000 


.100 


1091 


.141 


.000 


1091 


Student Effort 


0.451 


.075 


.092 


1091 


.059 


.050 


1091 


Academic Challenge 


0.571 


.012 


.094 


1091 


.100 


.001 


1091 


Student-Faculty Interaction 


0.433 


.037 


.093 


1090 


.090 


.003 


1090 


Support for Learners 


-0.168 


.347 


.090 


1089 


-.028 


.362 


1089 


Faculty Interactions 


0.619 


.004 


.096 


1091 


.121 


.000 


1091 


Class Assignments 


0.316 


.053 


.092 


1091 


.055 


.069 


1091 


Exposure to Diversity 


0.301 


.035 


.093 


1091 


.072 


.017 


1091 


Collaborative Learning 


0.524 


.010 


.095 


1090 


.098 


.001 


1090 


Information Technology 


0.195 


.142 


.091 


1090 


.058 


.056 


1090 


Mental Activities 


0.357 


.041 


.093 


1090 


.084 


.005 


1090 


School Opinions 


-0.081 


.638 


.089 


1082 


-.012 


.694 


1082 


Student Services 


-0.204 


.224 


.090 


1061 


-.047 


.126 


1061 


Academic Preparation 


0.873 


.001 


.089 


1083 


.127 


.000 


1083 


Gains in Academics 


0.106 


.064 


.092 


1082 


.055 


.072 


1082 


Gains in Personal Development 


-0.040 


.412 


.089 


1082 


-.016 


.602 


1082 


Gains in Vocational Goals 


-0.050 


.270 


.090 


1082 


-.012 


.697 


1082 



NOTE: Linear regression model (unstandardized betas) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, Developmental Math 
Placement Level, Sum of Risk Factors 
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Table B1 1 



Outcome: Credit Completion Ratio - Cumulative Y1 Y2 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff 


Sig. 


N 


Active and Collaborative Learning 


0.112 


.000 


.106 


1623 


.122 


.000 


1623 


Student Effort 


0.124 


.000 


.107 


1623 


.106 


.000 


1623 


Academic Challenge 


0.105 


.000 


.106 


1623 


.121 


.000 


1623 


Student-Faculty Interaction 


0.061 


.020 


.101 


1622 


.083 


.001 


1622 


Support for Learners 


-0.035 


.121 


.100 


1620 


-.045 


.068 


1620 


Faculty Interactions 


0.086 


.002 


.104 


1623 


.105 


.000 


1623 


Class Assignments 


0.094 


.000 


.110 


1623 


.114 


.000 


1623 


Exposure to Diversity 


0.018 


.987 


.099 


1623 


.031 


.210 


1623 


Collaborative Learning 


0.047 


.081 


.100 


1622 


.063 


.011 


1622 


Information Technology 


0.037 


.027 


.101 


1622 


.086 


.001 


1622 


Mental Activities 


0.074 


.001 


.104 


1622 


.106 


.000 


1622 


School Opinions 


-0.033 


.125 


.100 


1610 


-.040 


.111 


1610 


Student Services 


0.020 


.345 


.105 


1574 


-.001 


.985 


1574 


Academic Preparation 


0.103 


.001 


.104 


1611 


.128 


.000 


1611 


Gains in Academics 


0.024 


.001 


.105 


1610 


.078 


.002 


1610 


Gains in Personal Development 


-0.004 


.490 


.099 


1610 


-.030 


.233 


1610 


Gains in Vocational Goals 


-0.004 


.256 


.099 


1610 


-.019 


.455 


1610 


NOTE: Regression model (unstandardized betas) 














Controls: cohort, gender, race/ethnicity, age, part-time status Y1T1, developmental math 




placement level, sum of risk factors, cumulative credits attempted Y1Y2. 








Table B12 
















Outcome: Persistence, Fall-to-Fall Y1 Y2 (Cohorts 2002, 2003 only) 














Regression 




Correlation 


CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.758 


.052 


.118 


1229 


.059 


.037 


1229 


Student Effort 


0.738 


.081 


.117 


1229 


.058 


.041 


1229 


Academic Challenge 


0.576 


.130 


.117 


1229 


.038 


.183 


1229 


Student-Faculty Interaction 


-0.268 


.436 


.115 


1228 


-.019 


.498 


1228 


Support for Learners 


0.346 


.248 


.115 


1227 


.047 


.098 


1227 


Faculty Interactions 


-0.501 


.159 


.116 


1229 


-.042 


.145 


1229 


Class Assignments 


0.427 


.115 


.117 


1229 


.064 


.024 


1229 


Exposure to Diversity 


0.280 


.241 


.116 


1229 


.031 


.272 


1229 


Collaborative Learning 


0.685 


.053 


.118 


1228 


.066 


.021 


1228 


Information Technology 


0.527 


.019 


.120 


1228 


.063 


.027 


1228 


Mental Activities 


0.490 


.095 


.117 


1228 


.042 


.139 


1228 


School Opinions 


0.272 


.350 


.112 


1219 


.034 


.235 


1219 


Student Services 


0.577 


.041 


.120 


1195 


.079 


.006 


1195 


Academic Preparation 


0.271 


.518 


.112 


1220 


.021 


.456 


1220 


Gains in Academics 


0.294 


.002 


.121 


1219 


.121 


.000 


1219 


Gains in Personal Development 


0.048 


.553 


.111 


1219 


.040 


.158 


1219 


Gains in Vocational Goals 


0.136 


.075 


.114 


1219 


.062 


.031 


1219 



NOTE: Logistic regression model (Ft 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, placed in 
developmental math, placed in developmental English, sum of risk factors 
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Table B13 



Outcome: Attainment - Degree or Certificate by Year 3 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


1.192 


.017 


.203 


1623 


.101 


.000 


1623 


Student Effort 


-0.198 


.727 


.197 


1623 


-.008 


.759 


1623 


Academic Challenge 


1.009 


.050 


.201 


1623 


.069 


.005 


1623 


Student-Faculty Interaction 


1.232 


.006 


.205 


1622 


.110 


.000 


1622 


Support for Learners 


-0.522 


.210 


.198 


1620 


-.021 


.394 


1620 


Faculty Interactions 


1.331 


.004 


.206 


1623 


.108 


.000 


1623 


Class Assignments 


0.190 


.600 


.197 


1623 


.030 


.225 


1623 


Exposure to Diversity 


0.237 


.463 


.198 


1623 


.030 


.220 


1623 


Collaborative Learning 


0.909 


.040 


.202 


1622 


.088 


.000 


1622 


Information Technology 


0.355 


.231 


.199 


1622 


.064 


.010 


1622 


Mental Activities 


0.654 


.101 


.200 


1622 


.061 


.014 


1622 


School Opinions 


-0.395 


.325 


.199 


1610 


-.019 


.419 


1610 


Student Services 


-0.377 


.323 


.196 


1574 


-.022 


.383 


1574 


Academic Preparation 


1.306 


.019 


.204 


1611 


.080 


.001 


1611 


Gains in Academics 


0.013 


.919 


.198 


1610 


.009 


.718 


1610 


Gains in Personal Development 


-0.026 


.811 


.198 


1610 


.006 


.820 


1610 


Gains in Vocational Goals 


0.209 


.048 


.202 


1610 


.072 


.004 


1610 



NOTE: Logistic regression model (R 2 is Nagelkerke) 

Control Variables: Cohort, Gender, Race/Ethnicity, Age, Part-Time Y1T1, placed in 
developmental math, placed in developmental English, sum of risk factors 



Table B14 

Outcome: Credit Completion Ratios in Term CCSSE Administered if Spring of First Academic 
Year 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.160 


.006 


.052 


975 


.103 


.001 


975 


Student Effort 


0.198 


.001 


.056 


975 


.105 


.001 


975 


Academic Challenge 


0.220 


.000 


.049 


975 


.139 


.000 


975 


Student-Faculty Interaction 


0.113 


.022 


.050 


974 


.082 


.010 


974 


Support for Learners 


0.019 


.647 


.045 


973 


-.006 


.855 


973 


Faculty Interactions 


0.153 


.003 


.053 


975 


.105 


.001 


975 


Class Assignments 


0.143 


.000 


.047 


975 


.117 


.000 


975 


Exposure to Diversity 


-0.005 


.875 


.045 


975 


.003 


.936 


975 


Collaborative Learning 


0.081 


.134 


.047 


974 


.049 


.123 


974 


Information Technology 


0.086 


.006 


.052 


974 


.106 


.001 


974 


Mental Activities 


0.142 


.001 


.056 


974 


.116 


.000 


974 


School Opinions 


0.022 


.580 


.045 


968 


.005 


.874 


968 


Student Services 


0.021 


.570 


.054 


939 


-.018 


.575 


939 


Academic Preparation 


0.244 


.000 


.061 


969 


.147 


.000 


969 


Gains in Academics 


0.038 


.004 


.053 


968 


.064 


.045 


968 


Gains in Personal Development 


0.009 


.427 


.045 


968 


.001 


.966 


968 


Gains in Vocational Goals 


0.008 


.477 


.045 


968 


.018 


.585 


968 



NOTE: Regression model (unstandardized betas) 

Controls: cohort, gender, race/ethnicity, age, part-time status when CCSSE administered, 
developmental math placement level, sum of risk factors 
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Table B15 



Outcome: Cumulative GPA in year CCSSE Administered (if Year 1) 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.432 


.013 


.108 


975 


.095 


.003 


975 


Student Effort 


0.573 


.001 


.113 


975 


.098 


.002 


975 


Academic Challenge 


0.523 


.001 


.112 


975 


.108 


.001 


975 


Student-Faculty Interaction 


0.244 


.101 


.105 


974 


.056 


.079 


974 


Support for Learners 


-0.044 


.720 


.103 


973 


-.049 


.128 


973 


Faculty Interactions 


0.398 


.010 


.109 


975 


.093 


.004 


975 


Class Assignments 


0.405 


.000 


.114 


975 


.098 


.002 


975 


Exposure to Diversity 


0.057 


.566 


.103 


975 


.018 


.581 


975 


Collaborative Learning 


0.187 


.252 


.104 


974 


.034 


.288 


974 


Information Technology 


0.165 


.079 


.105 


974 


.073 


.023 


974 


Mental Activities 


0.336 


.008 


.109 


974 


.089 


.005 


974 


School Opinions 


-0.004 


.975 


.102 


968 


-.030 


.356 


968 


Student Services 


0.032 


.777 


.112 


939 


-.039 


.229 


939 


Academic Preparation 


0.647 


.000 


.114 


969 


.139 


.000 


969 


Gains in Academics 


0.057 


.148 


.104 


968 


.004 


.899 


968 


Gains in Personal Development 


-0.043 


.201 


.104 


968 


-.078 


.015 


968 


Gains in Vocational Goals 


-0.049 


.132 


.104 


968 


-.060 


.062 


968 



NOTE: Regression model (unstandardized betas) 

Controls: cohort, gender, race/ethnicity, age, part-time status when CCSSE administered, 
developmental math placement level, sum of risk factors 



Table B16 

Outcome: Credit Completion Ratios in Term CCSSE Administered if Spring of Second Academic 
Year 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.047 


.403 


.104 


547 


.062 


.148 


547 


Student Effort 


0.051 


.444 


.103 


547 


.023 


.585 


547 


Academic Challenge 


0.042 


.471 


.103 


547 


.066 


.125 


547 


Student-Faculty Interaction 


-0.024 


.645 


.103 


547 


.024 


.580 


547 


Support for Learners 


-0.067 


.151 


.106 


546 


-.038 


.378 


546 


Faculty Interactions 


0.021 


.689 


.103 


547 


.060 


.162 


547 


Class Assignments 


0.009 


.841 


.102 


547 


.005 


.898 


547 


Exposure to Diversity 


0.025 


.505 


.103 


547 


.025 


.558 


547 


Collaborative Learning 


-0.020 


.689 


.103 


547 


.008 


.850 


547 


Information Technology 


-0.038 


.285 


.104 


547 


-.013 


.766 


547 


Mental Activities 


0.039 


.373 


.104 


547 


.068 


.112 


547 


School Opinions 


-0.082 


.064 


.106 


541 


-.046 


.290 


541 


Student Services 


0.012 


.793 


.094 


534 


-.026 


.546 


534 


Academic Preparation 


0.099 


.128 


.104 


541 


.099 


.021 


541 


Gains in Academics 


0.009 


.568 


.100 


541 


.052 


.231 


541 


Gains in Personal Development 


0.004 


.762 


.100 


541 


.029 


.497 


541 


Gains in Vocational Goals 


-0.008 


.510 


.100 


541 


.016 


.711 


541 



NOTE: Regression model (unstandardized betas) 

Controls: cohort, gender, race/ethnicity, age, part-time status when CCSSE administered, 
developmental math placement level, sum of risk factors 



121 




Table B17 



Outcome: Cumulative GPA in year CCSSE Administered (if Year 2) 



Regression Correlation 



CCSSE Predictor 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.645 


.000 


.191 


548 


.208 


.000 


548 


Student Effort 


0.400 


.035 


.173 


548 


.114 


.008 


548 


Academic Challenge 


0.391 


.018 


.175 


548 


.151 


.000 


548 


Student-Faculty Interaction 


0.181 


.222 


.168 


548 


.104 


.105 


548 


Support for Learners 


-0.235 


.081 


.171 


547 


-.039 


.367 


547 


Faculty Interactions 


0.346 


.021 


.174 


548 


.153 


.000 


548 


Class Assignments 


0.258 


.033 


.173 


548 


.077 


.070 


548 


Exposure to Diversity 


0.314 


.003 


.180 


548 


.121 


.005 


548 


Collaborative Learning 


0.329 


.019 


.174 


548 


.131 


.002 


548 


Information Technology 


0.089 


.387 


.167 


548 


.073 


.088 


548 


Mental Activities 


0.344 


.007 


.177 


548 


.149 


.000 


548 


School Opinions 


-0.165 


.196 


.170 


542 


-.022 


.605 


542 


Student Services 


-0.074 


.564 


.170 


535 


-.019 


.653 


535 


Academic Preparation 


0.413 


.027 


.175 


542 


.179 


.000 


542 


Gains in Academics 


0.065 


.127 


.171 


542 


.098 


.023 


542 


Gains in Personal Development 


-0.053 


.133 


.171 


542 


-.030 


.480 


542 


Gains in Vocational Goals 


-0.070 


.037 


.174 


542 


-.029 


.495 


542 



NOTE: Regression model (unstandardized betas) 

Controls: cohort, gender, race/ethnicity, age, part-time status when CCSSE administered, 
developmental math placement level, sum of risk factors 
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Appendix C: HSS Consortium Institutions Validation Study Results 



Table Cl 



Outcome: Cumulative GPA 



CCSSE Predictor 


Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.064 


.000 


.074 


3198 


.082 


.000 


3265 


Student Effort 


0.081 


.000 


.077 


3198 


.119 


.000 


3265 


Academic Challenge 


0.082 


.000 


.077 


3198 


.103 


.000 


3265 


Student-Faculty Interaction 


0.066 


.000 


.075 


3198 


.077 


.000 


3265 


Support for Learners 


-0.019 


.261 


.071 


3194 


-.020 


.249 


3260 


Faculty Interactions 


0.092 


.000 


.079 


3198 


.117 


.000 


3265 


Class Assignments 


0.043 


.012 


.072 


3197 


.050 


.004 


3264 


Exposure to Diversity 


0.039 


.024 


.072 


3197 


.045 


.010 


3263 


Collaborative Learning 


0.017 


.309 


.071 


3197 


.011 


.582 


3264 


Information Technology 


0.044 


.011 


.072 


3195 


.046 


.009 


3262 


Mental Activities 


0.082 


.000 


.077 


3197 


.095 


.000 


3264 


School Opinions 


0.000 


.991 


.071 


3161 


-.002 


.888 


3227 


Student Services 


-0.038 


.029 


.069 


3117 


-.020 


.253 


3174 


Academic Preparation 


0.093 


.000 


.079 


3164 


.124 


.000 


3231 


Gains in Academics 


0.041 


.019 


.072 


3159 


..057 


.001 


3221 


Gains in Personal Development 


-0.010 


.553 


.071 


3155 


-.002 


.911 


3218 


Gains in Vocational Goals 


-0.010 


.553 


.071 


3155 


.020 


.256 


3223 



Table C2 

Outcome: First to Second Term Persistence 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.110 


.000 


.012 


3127 


.110 


.000 


3194 


Student Effort 


0.046 


.011 


.003 


3127 


.048 


.006 


3194 


Academic Challenge 


0.038 


.034 


.002 


3127 


.038 


.030 


3194 


Student-Faculty Interaction 


0.095 


.000 


.009 


3127 


.093 


.000 


3194 


Support for Learners 


0.056 


.002 


.004 


3122 


.052 


.003 


3189 


Faculty Interactions 


0.082 


.000 


.007 


3127 


.081 


.000 


3194 


Class Assignments 


0.104 


.000 


.011 


3126 


.103 


.000 


3193 


Exposure to Diversity 


0.070 


.000 


.005 


3125 


.067 


.000 


3193 


Collaborative Learning 


0.090 


.000 


.009 


3127 


.090 


.000 


3194 


Information Technology 


0.050 


.005 


.003 


3125 


.047 


.008 


3193 


Mental Activities 


0.019 


.290 


.001 


3127 


.019 


.288 


3194 


School Opinions 


0.043 


.016 


.002 


3092 


.041 


.020 


3159 


Student Services 


0.054 


.003 


.004 


3053 


.055 


.002 


3110 


Academic Preparation 


0.042 


.019 


.002 


3095 


.044 


.013 


3162 


Gains in Academics 


0.094 


.000 


.009 


3091 


.092 


.000 


3154 


Gains in Personal Development 


0.056 


.000 


.008 


3087 


.083 


.000 


3151 


Gains in Vocational Goals 


0.071 


.000 


.005 


3092 


.068 


.000 


3155 
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Table C3 



Outcome: First to Third Term Persistence 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.113 


.000 


.014 


3125 


.112 


.000 


3193 


Student Effort 


0.038 


.037 


.003 


3125 


.038 


.034 


3193 


Academic Challenge 


0.053 


.003 


004 


3125 


.054 


.002 


3193 


Student-Faculty Interaction 


0.093 


.000 


.010 


3125 


.094 


.000 


3193 


Support for Learners 


0.076 


.000 


.007 


3121 


.070 


.000 


3188 


Faculty Interactions 


0.079 


.000 


.007 


3125 


.081 


.000 


3193 


Class Assignments 


0.087 


.000 


.009 


3124 


.084 


.000 


3192 


Exposure to Diversity 


0.042 


.020 


.003 


3124 


.045 


.010 


3191 


Collaborative Learning 


0.108 


.000 


.013 


3125 


.106 


.000 


3193 


Information Technology 


0.050 


.005 


.004 


3123 


.049 


.005 


3191 


Mental Activities 


0.036 


.044 


.002 


3125 


.036 


.041 


3193 


School Opinions 


0.063 


.001 


.005 


3090 


.058 


.001 


3157 


Student Services 


0.069 


.010 


.006 


3050 


.069 


.000 


3107 


Academic Preparation 


0.047 


.000 


.003 


3093 


.050 


.005 


3160 


Gains in Academics 


0.121 


.000 


.015 


3082 


.114 


.000 


3152 


Gains in Personal Development 


0.104 


.000 


.012 


3086 


.097 


.000 


3149 


Gains in Vocational Goals 


0.075 


.000 


.007 


3090 


.074 


.000 


3153 



Table C4 

Outcome: Total Credit Hours Taken 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.181 


.000 


.045 


3211 


.178 


.000 


3279 


Student Effort 


0.123 


.000 


.028 


3211 


.114 


.000 


3279 


Academic Challenge 


0.132 


.000 


.030 


3211 


.131 


.000 


3279 


Student-Faculty Interaction 


0.167 


.000 


.041 


3211 


.175 


.000 


3279 


Support for Learners 


0.140 


.000 


.032 


3207 


.124 


.000 


3274 


Faculty Interactions 


0.131 


.000 


.030 


3211 


.136 


.000 


3279 


Class Assignments 


0.160 


.000 


.038 


3210 


.153 


.000 


3278 


Exposure to Diversity 


0.115 


.000 


.026 


3210 


.130 


.000 


3278 


Collaborative Learning 


0.170 


.000 


.042 


3210 


.166 


.000 


3278 


Information Technology 


0.117 


.000 


.026 


3209 


.122 


.000 


3277 


Mental Activities 


0.103 


.000 


.024 


3210 


.104 


.000 


3278 


School Opinions 


0.127 


.000 


.028 


3174 


.113 


.000 


3241 


Student Services 


0.141 


.000 


.033 


3131 


.134 


.000 


3188 


Academic Preparation 


0.120 


.000 


.027 


3178 


.121 


.000 


3245 


Gains in Academics 


0.207 


.000 


.055 


3172 


.191 


.000 


3236 


Gains in Personal Development 


0.179 


.000 


.044 


3169 


.159 


.000 


3232 


Gains in Vocational Goals 


0.131 


.000 


.029 


3175 


.116 


.000 


3237 
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Table C5 



Outcome: Number of Terms Enrolled 



CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.118 


.000 


.037 


3211 


.121 


.000 


3279 


Student Effort 


0.050 


.004 


.025 


3211 


.065 


.000 


3279 


Academic Challenge 


0.059 


.001 


.026 


3211 


.066 


.000 


3279 


Student-Faculty Interaction 


0.112 


.000 


.035 


3211 


.116 


.000 


3279 


Support for Learners 


0.074 


.000 


.028 


3207 


.060 


.001 


3274 


Faculty Interactions 


0.092 


.000 


.031 


3211 


.104 


.000 


3279 


Class Assignments 


0.098 


.000 


.032 


3210 


.092 


.000 


3278 


Exposure to Diversity 


0.076 


.000 


.028 


3210 


.079 


.000 


3278 


Collaborative Learning 


0.101 


.000 


.033 


3210 


.092 


.000 


3278 


Information Technology 


0.041 


.019 


.025 


3209 


.038 


.029 


3277 


Mental Activities 


0.061 


.000 


.026 


3210 


.066 


.000 


3278 


School Opinions 


0.060 


.001 


.027 


3174 


.049 


.006 


3241 


Student Services 


0.075 


.000 


.030 


3131 


.079 


.000 


3188 


Academic Preparation 


0.002 


.907 


.023 


3178 


.016 


.355 


3245 


Gains in Academics 


0.142 


.000 


.043 


3172 


.140 


.000 


3236 


Gains in Personal Development 


0.134 


.000 


.041 


3169 


.124 


.000 


3232 


Gains in Vocational Goals 


0.096 


.000 


.033 


3175 


.092 


.000 


3237 



Table C6 

Outcome: Average Credit Hours Taken 


CCSSE Predictor 




Regression 




Correlation 


Beta 


Sig. 


R 2 


N 


Coeff. 


Sig. 


N 


Active and Collaborative Learning 


0.158 


.000 


.102 


3211 


.149 


.000 


3279 


Student Effort 


0.152 


.000 


.099 


3211 


.113 


.000 


3279 


Academic Challenge 


0.154 


.000 


.100 


3211 


.141 


.000 


3279 


Student-Faculty Interaction 


0.142 


.000 


.097 


3211 


.150 


.000 


3279 


Support for Learners 


0.138 


.000 


.096 


3207 


.131 


.000 


3274 


Faculty Interactions 


0.115 


.000 


.090 


3211 


.106 


.000 


3279 


Class Assignments 


0.161 


.000 


.103 


3210 


.157 


.000 


3278 


Exposure to Diversity 


0.091 


.000 


.085 


3210 


.114 


.000 


3278 


Collaborative Learning 


0.165 


.000 


.104 


3210 


.170 


.000 


3278 


Information Technology 


0.155 


.000 


.101 


3209 


.167 


.000 


3277 


Mental Activities 


0.094 


.000 


.080 


3210 


.087 


.000 


3278 


School Opinions 


0.130 


.000 


.094 


3174 


.124 


.000 


3241 


Student Services 


0.139 


.000 


.095 


3131 


.120 


.000 


3188 


Academic Preparation 


0.235 


.000 


.131 


3178 


.214 


.000 


3245 


Gains in Academics 


0.158 


.000 


.102 


3172 


.133 


.000 


3236 


Gains in Personal Development 


0.117 


.000 


.091 


3169 


.095 


.000 


3232 


Gains in Vocational Goals 


0.072 


.000 


.082 


3175 


.053 


.003 


3237 
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Appendix D: CCSSE Constructs 



Benchmark Descriptions for 

the Community College Survey of Student Engagement Data 



Active and Collaborative Learning Benchmark composed of seven survey items. A four- 

item response scale (Never, Sometimes, Often, Very 
often) corresponds to the following Active and 
Collaborative Learning college activities: 

• Asked questions in class or contributed to class 
discussions 

• Made a class presentation 

• Worked with other students on projects during class 

• Worked with classmates outside of class to prepare 
class assignments 

• Tutored or taught other students (paid or voluntary) 

• Participated in a community-based project as a part 
of a regular course 

• Discussed ideas from your readings or classes with 
others outside of class (students, family members, 
co-workers, etc.) 

Student Effort Benchmark composed of eight survey items. A four- 

item response scale (Never, Sometimes, Often, Very 
often) corresponds to the following Student Effort 
related college activities: 

• Prepared two or more drafts of a paper or 
assignment before turning it in 

• Worked on a paper or project that required 
integrating ideas or information from various 
sources 

• Come to class without completing readings or 
assignments 

A five-item response scale (None, Between 1 and 4, 
Between 5 and 10, Between 1 1 and 20, More than 20) is 
used for the following academic preparation item: 

• Number of books read on your own (not assigned) 
for personal enjoyment or academic enrichment 

A six-item response scale (None, 1-5 hours, 6-10 hours, 
11-20 hours, 21-30 hours, More than 30 hours) is used 
for the following time allotment item: 

• Preparing for class (studying, reading, writing, 
rehearsing, doing homework, or other activities 
related to your program) 
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A four-item response scale (Don’t Know/N.A., 
Rarely/never, Sometimes, Often) is used for the 





following student services items: 



Academic Challenge 



Student-Faculty Interaction 



• Frequency: peer or other tutoring 

• Frequency: skill labs (writing, math, etc.) 

• Frequency: computer lab 

Benchmark composed of ten survey items. A four-item 
response scale (Never, Sometimes, Often, Very often) is 
used for the following Academic Challenge related 
college activity: 

• Worked harder than you thought you could to meet 
an instructor’s standards or expectations 

A four-item response scale ( Very little, Some, Quite a 
bit, Very much) is used for the following mental activity 
items: 



• Analyzing the basic elements of an idea, 
experience, or theory 

• Synthesizing and organizing ideas, information, or 
experiences in new ways 

• Making judgments about the value or soundness of 
information, arguments, or methods 

• Applying theories or concepts to practical problems 
or in new situations 

• Using information you have read or heard to 
perform a new skill 

A five-item response scale (None, Between 1 and 4, 
Between 5 and 10, Between 1 1 and 20, More than 20) is 
used for the following academic preparation items: 

• Number of assigned textbooks, manuals, books, or 
book-length packs of course readings 

• Number of written papers or reports of any length 

A seven-item response scale (Ranging from 1 to 7, with 
scale anchors described: (1) Extremely easy (7) 
Extremely challenging) is used for the following exam 
item: 

• Mark the box that best represents the extent to 
which your examinations during the current school 
year have challenged you to do your best work at 
this college 



A four-item response scale ( Very little, Some, Quite a 
bit, Very much) is used for the following college opinion 
item: 

• Encouraging you to spend significant amounts of 
time studying 

Benchmark composed of six survey items. A four-item 
response scale (Never, Sometimes, Often, Very often) 
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is used for the following Student-Faculty Interaction 
related college activities: 

• Used email to communicate with an instructor 

• Discussed grades or assignments with an instructor 

• Talked about career plans with an instructor or 
advisor 

• Discussed ideas from your readings or classes with 
instructors outside of class 

• Received prompt feedback (written or oral) from 
instructors on your performance 

• Worked with instructors on activities other than 
coursework 

Support for Learners Benchmark composed of seven survey items. A four- 

item response scale (Very little, Some, Quite a bit, Very 
much) is used for the following college opinion items: 

• Providing the support you need to help you succeed 
at this college 

• Encouraging contact among students from different 
economic, social, and racial or ethnic backgrounds 

• Helping you cope with your non-academic 
responsibilities (work, family, etc.) 

• Providing the support you need to thrive socially 

• Providing the financial support you need to afford 
your education 

A four-item response scale (Don’t know/N.A., 
Rarely/never, Sometimes, Often) is used for the 
following student services items: 

• Frequency: Academic advising/planning 

• Frequency: Career counseling 



Definitions of Item Clusters for 

the Community College Survey of Student Engagement Data 



Faculty Interactions Indicator composed of six survey items. A four-item 

response scale (Never, Sometimes, Often, Very Often) 

is used for the following college activities: 

• Asked questions in class or contributed to class 
discussions 

• Discussed grades or assignments with an instructor 

• Talked about career plans with an instructor or 
advisor 

• Discussed ideas from your readings or classes with 
instructors outside of class 

• Received prompt feedback (written or oral) from 
instructors on your performance 

• Worked with instructors on activities other than 
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coursework 



Class Assignments 



Exposure to Diversity 



Collaborative Learning 



Information Technology 



Mental Activities 



Indicator composed of three survey items. A four-item 
response scale (Never, Sometimes, Often, Very Often) 
is used for the following college activities: 

• Made a class presentation 

• Prepared two or more drafts of a paper or 
assignment before turning it in 

• Worked on a paper or project that required 
integrating ideas or information from various 
sources 

Indicator composed of three survey items. A four-item 
response scale (Never, Sometimes, Often, Very Often) 
is used for the following college activities: 

• Discussed ideas from your readings or classes with 
others outside of class (students, family members, 
co-workers, etc.) 

• Had serious conversations with students of a 
different race or ethnicity other than your own 

• Had serious conversations with students who differ 
from you in terms of their religious beliefs, political 
opinions, or personal values 

Indicator composed of four survey items. A four-item 
response scale (Never, Sometimes, Often, Very Often) 
is used for the following college activities: 

• Worked with other students on projects during class 

• Worked with classmates outside of class to prepare 
class assignments 

• Tutored or taught other students (paid or voluntary) 

• Participated in a community-based project as a part 
of a regular course 

Indicator composed of two survey items. A four-item 
response scale (Never, Sometimes, Often, Very Often) 
is used for the following college activities: 

• Used the internet or instant messaging to work on 
an assignment 

• Used email to communicate with an instructor 

Indicator composed of six survey items. A four-item 
response scale (Never, Sometimes, Often, Very Often) 
is used for the following college activity: 

• Worked harder than you thought you could to meet 
an instructor’s standards or expectations 



A four-item response scale ( Very little, Some, Quite a 
bit, Very much) is used for the following mental activity 
items: 
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School Opinions 



Student Services 



Academic Preparation 



• Analyzing the basic elements of an idea, 
experience, or theory 

• Synthesizing and organizing ideas, information, or 
experiences in new ways 

• Making judgments about the value or soundness of 
information, arguments, or methods 

• Applying theories or concepts to practical problems 
or in new situations 

• Using information you have read or heard to 
perform a new skill 

Indicator composed of six survey items. A four-item 
response scale ( Very little, Some, Quite a bit, Very 
Much) is used for the following college opinion items: 

• Encouraging you to spend significant amounts of 
time studying 

• Providing the support you need to help you succeed 
at this college 

• Encouraging contact among students from different 
economic, social, and racial or ethnic backgrounds 

• Helping you cope with your non-academic 
responsibilities (work, family, etc.) 

• Providing the support you need to thrive socially 

• Providing the financial support you need to afford 
your education 

Indicator composed of five survey items. A four-item 
response scale (Don’t Know/N.A., Rarely/never, 
Sometimes, Often) is used for the following student 
services items: 

• Frequency: Academic advising/planning 

• Frequency: Career counseling 

• Frequency: Peer or other tutoring 

• Frequency: Skill labs (writing, math, etc.) 

• Frequency: Computer lab 

Indicator composed of four survey items. A five-item 
response scale (None, Between 1 and 4, Between 5 and 
10, Between 1 1 and 20, More than 20) is used for the 
following academic preparation items: 

• Number of assigned textbooks, manuals, books, or 
book-length packs of course readings 

• Number of written papers or reports of any length 

A seven-item response scale ( Ranging from 1 to 7, with 
scale anchors described: ( 1 ) Extremely easy (7) 
Extremely challenging) is used for the following exam 
item: 

• Mark the box that best represents the extent to 
which your examinations during the current school 
year have challenged you to do your best work at 
this college 
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A six-item response scale (None, 1-5 hours, 6-10 hours, 
11-20 hours, 21-30 hours, More than 30 hours) is used 
for the following time allotment item: 

• Preparing for class (studying, reading, writing, 
rehearsing, doing homework, or other activities 
related to your program) 



Definitions of Perceived Gain Items for 

the Community College Survey of Student Engagement Data 



Gains in Academics Gain index based on five survey items. A four-item 

response scale ( Very little, Some, Quite a bit, Very 
much) is used for the following academic gain items: 

• Acquiring a broad general education 

• Writing clearly and effectively 

• Speaking clearly and effectively 

• Thinking critically and analytically 

• Solving numerical problems 

Gains in Personal Development Gain index based on four survey items. A four-item 

response scale ( Very little, Some, Quite a bit, Very 
much) is used for the following personal development 
gain items: 

• Understanding yourself 

• Understanding people of other racial and ethnic 
backgrounds 

• Developing a personal code of values and ethics 

• Contributing to the welfare of your community 

Gains in Vocational Goals Gain index based on three survey items. A four-item 

response scale ) Very little, Some, Quite a bit, Very 
much) is used for the following vocational goal gain 
items: 

• Acquiring job or work-related knowledge and skills 

• Developing clearer career goals 

• Gaining information about career opportunities 
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Appendix E: Study Variables 



Florida Community College System Variables 
Identifiers: 

Student Identification Number 
Community College Attended 
Year 
Term 

Student Characteristics: 

Gender 

Race 

Age at Entry* 

Residence 
Citizenship 
Nationality 
Disability Flag 
Limited English Flag 
Incarceration Flag 

Descriptors from CCSSE Responses: 

Highest Degree Attained 

Goal for Attendance (Taking Courses for Personal Interest) 

Number of Risk Factors (Standard CCSSE Calculation for Students “At Risk)* 

Educational Background: 

High School Name 

High School Type 

High School Location 

High School Graduation Type 

Time from High School Graduation* 

Transfer Institution 

Test Scores: 

CPT (Reading, Sentence Skills, Elementary Algebra) 

CPT Testing Dates 
SAT (Verbal, Math) 

SAT Testing Dates 

ACT (Reading, English, Math) 

ACT Testing Dates 

CLAST First Testing (Reading, Language Arts, Math, Essay)* 

CLAST First Testing Dates* 

CLAST Latest Testing (Reading, Language Arts, Math, Essay)* 

CLAST Latest Testing Dates* 

Remedial Reading Flag (Indicates Placement Level)* 

Remedial Writing Flag (Indicates Placement Level)* 

Remedial Math Flag (Indicates Placement Level)* 

Total CPT Score* 
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Enrollment Status: 



First Time in College Flag 
Transfer Status 
Admit Status 
Entry Date 
Class Level 

Term Variables (Repeated as Needed): 

Term Clock Hour Load 
Term Clock Hours Earned 
Term Credit Hour Load 
Term Credit Hours Earned 

Term Credit Equivalent Hour Load (Combines Clock and Credit Hours)* 

Term Credit Equivalent Hours Earned (Combines Clock and Credit Hours)* 

Term Grade Points 

Term GPA Hours 

Total Grade Points 

Part-time Indicator 

Dual Enrollment Flag (from Course File) 

Program CIP Cluster (up to three)* 

Program CIP Code (up to three)* 

Award or Certification Sought (up to three)* 

Pell Grant Award* 

Federal Need-Based Aid* 

Federal Loans* 

State Need-Based Aid* 

State Merit-Based Aid* 

Other Loan* 

Other Scholarship* 

Total All Aid Sources* 

Term GPA* 

Cumulative GPA* 

Term Credit Completion Ratio* 

Cumulative Credit Completion Ratio* 

Term Percent Courses Completed with Grade of “C” or Better* 

Cumulative Percent Courses Competed with Grade of “C” or Better* 
Enrolled (Persistence) Flag* 

Award Flag* 

Award CIP Cluster (up to three)* 

Award CIP Code* 

Course Enrollment Data 

Student Identification Number 
Community College ID 
Year 
Term 

Course Number 
Section Number 
Grade 
Hours Type 
Credits 

Credit Equivalent Hours (Combined Clock and Credit Hours)* 

Dual Enrollment Flag 
Gatekeeper Course Flag* 
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Developmental Course Flag 
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Appendix F: Participating Institutions 



Participating Florida Community College System Institutions 

Brevard Community College 
Broward Community College 
Central Florida Community College 
Chipola College 

Daytona Beach Community College 
Edison College 

Florida Community College at Jacksonville 
Florida Keys Community College 
Gulf Coast Community College 
Hillsborough Community College 
Indian River Community College 
Lake City Community College 
Lake-Sumter Community College 
Manatee Community College 
Miami Dade College 
North Florida Community College 
Okaloosa-Walton Community College 
Palm Beach Community College 
Pasco-Hernando Community College 
Pensacola Junior College 
Polk Community College 
St. Petersburg College 
Santa Fe Community College 
Seminole Community College 
St. Johns River Community College 
South Florida Community College 
Tallahassee Community College 
Valencia Community College 
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Participating Achieving the Dream Colleges 



Albuquerque TVI 

Brookhaven College 

Broward Community College 

Capital Community College 

Dona Ana Branch Community College - NMSU 

Durham Technical Community College 

El Paso Community College 

Guilford Technical Community College 

Housatonic Community College 

Houston Community College System 

Northwest Vista College 

Norwalk Community College 

Palo Alto College 

Patrick Henry Community College 

Paul D. Camp Community College 

San Antonio College 

San Juan College 

Santa Fe Community College 
Southwest Texas Junior College 
St. Philip's College 
Tidewater Community College 
University of New Mexico - Gallup 
Valencia Community College 
Wayne Community College 
Zane State College 



Participating HSS Consortium Institutions 



Austin Community College 

Brazosport College 

Broward Community College 

Central Arizona College 

Coastal Bend College 

College of the Mainland 

Community College of Denver 

Estrella Mountain Community College 

Galveston College 

Howard College 

Miami Dade College 

New Mexico Junior College 

New Mexico State University at Alamogordo 

North Harris Montgomery Community College District 

North Lake College 

Pasco-Hernando Community College 

Phoenix College 

Richland College 

Southwest Texas Junior College 

Valencia Community College 
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